Jul 15 12:18:11.744657 ip-10-0-135-6 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 15 12:18:11.744667 ip-10-0-135-6 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 15 12:18:11.744674 ip-10-0-135-6 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 15 12:18:11.744884 ip-10-0-135-6 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 15 12:18:21.508780 ip-10-0-135-6 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 15 12:18:21.508798 ip-10-0-135-6 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d8af7638a5d146dd827a1fc21178fd84 --
Jul 15 12:20:09.742242 ip-10-0-135-6 systemd[1]: Starting Kubernetes Kubelet...
Jul 15 12:20:10.072479 ip-10-0-135-6 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 12:20:10.072479 ip-10-0-135-6 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 15 12:20:10.072479 ip-10-0-135-6 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 12:20:10.072479 ip-10-0-135-6 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 15 12:20:10.072479 ip-10-0-135-6 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 15 12:20:10.073974 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.073873    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 15 12:20:10.077352 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077338    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:20:10.077352 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077353    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077357    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077360    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077364    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077368    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077371    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077374    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077376    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077379    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077381    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077384    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077391    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077394    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077396    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077399    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077401    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077403    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077407    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077411    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:20:10.077412 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077414    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077417    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077419    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077422    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077425    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077428    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077431    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077433    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077436    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077438    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077441    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077443    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077445    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077448    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077450    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077452    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077455    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077458    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077460    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077463    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:20:10.077854 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077465    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077468    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077470    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077473    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077475    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077478    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077480    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077482    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077485    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077487    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077490    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077493    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077497    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077500    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077502    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077505    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077507    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077510    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077512    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:20:10.078332 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077516    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077519    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077522    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077525    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077527    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077530    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077532    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077534    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077537    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077539    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077542    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077545    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077547    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077550    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077553    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077555    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077559    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077561    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077564    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077566    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:20:10.078775 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077568    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077571    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077574    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077576    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077579    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077582    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077585    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077588    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077590    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077593    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077595    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077598    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077600    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077603    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077605    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077607    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077610    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077612    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077614    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.077617    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:20:10.079238 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078838    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078846    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078849    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078853    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078856    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078859    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078863    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078866    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078869    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078873    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078876    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078879    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078882    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078885    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078887    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078891    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078893    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078896    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078898    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:20:10.079733 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078901    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078903    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078906    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078909    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078911    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078914    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078916    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078918    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078921    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078937    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078940    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078942    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078945    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078948    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078952    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078956    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078958    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078961    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078963    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078966    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:20:10.080211 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078969    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078972    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078975    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078978    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078981    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078984    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078987    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078989    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078992    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078994    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.078997    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079000    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079002    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079004    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079007    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079010    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079012    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079015    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079017    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:20:10.080695 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079019    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079022    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079026    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079029    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079032    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079034    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079037    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079039    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079042    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079044    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079047    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079049    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079052    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079054    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079056    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079060    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079062    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079065    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079069    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079071    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:20:10.081192 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079074    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079079    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079082    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079084    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079087    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079089    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079092    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079095    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079097    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079100    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079103    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079105    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079107    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079110    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079112    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079115    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079117    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079119    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079122    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079124    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:20:10.081664 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079127    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079490    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079496    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079500    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079502    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079505    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079508    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079511    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079513    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079517    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079519    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079522    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079527    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079532    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079535    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079538    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079541    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079554    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079557    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:20:10.082146 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079560    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079563    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079565    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079568    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079570    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079573    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079575    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079578    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079599    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079602    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079604    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079607    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079609    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079612    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079614    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079618    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079621    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079623    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079626    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079628    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:20:10.082619 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079631    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079634    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079636    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079640    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079644    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079648    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079654    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079658    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079661    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079665    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079669    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079672    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079675    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079677    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079680    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079682    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079685    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079687    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079690    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079692    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:20:10.083101 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079696    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079698    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079701    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079703    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079706    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079709    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079711    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079715    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079717    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079720    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079722    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079724    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079727    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079730    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079732    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079735    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079738    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079741    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079745    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079749    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:20:10.083597 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079751    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079753    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079756    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079758    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079761    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079763    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079765    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079768    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079771    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079774    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079778    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079780    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079783    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079786    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079788    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079791    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079793    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079796    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079798    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:20:10.084131 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079801    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.079804    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079870    2576 flags.go:64] FLAG: --address="0.0.0.0"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079879    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079889    2576 flags.go:64] FLAG: --anonymous-auth="true"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079899    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079905    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079910    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079916    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079921    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079936    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079942    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079945    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079949    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079952    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079955    2576 flags.go:64] FLAG: --cgroup-root=""
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079958    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079961    2576 flags.go:64] FLAG: --client-ca-file=""
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079963    2576 flags.go:64] FLAG: --cloud-provider="external"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079967    2576 flags.go:64] FLAG: --cluster-dns="[]"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079971    2576 flags.go:64] FLAG: --cluster-domain=""
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079973    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079976    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079979    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 15 12:20:10.084576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079983    2576 flags.go:64] FLAG: --container-log-max-files="5"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079987    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079990    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079993    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079996    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.079999    2576 flags.go:64] FLAG: --contention-profiling="false"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080002    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080005    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080009    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080012    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080016    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080019    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080022    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080024    2576 flags.go:64] FLAG: --enable-load-reader="false"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080027    2576 flags.go:64] FLAG: --enable-server="true"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080030    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080034    2576 flags.go:64] FLAG: --event-burst="100"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080038    2576 flags.go:64] FLAG: --event-qps="50"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080041    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080046    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080049    2576 flags.go:64] FLAG: --eviction-hard=""
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080053    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080056    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080058    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080062    2576 flags.go:64] FLAG: --eviction-soft=""
Jul 15 12:20:10.085160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080064    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080067    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080070    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080073    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080076    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080079    2576 flags.go:64] FLAG: --fail-swap-on="true"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080081    2576 flags.go:64] FLAG: --feature-gates=""
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080085    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080088    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080091    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080094    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080097    2576 flags.go:64] FLAG: --healthz-port="10248"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080100    2576 flags.go:64] FLAG: --help="false"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080102    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-135-6.ec2.internal"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080106    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080109    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080111    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080115    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080118    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080121    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080123    2576 flags.go:64] FLAG: --image-service-endpoint=""
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080126    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080129    2576 flags.go:64] FLAG: --kube-api-burst="100"
Jul 15 12:20:10.085731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080132    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080135    2576 flags.go:64] FLAG: --kube-api-qps="50"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080138    2576 flags.go:64] FLAG: --kube-reserved=""
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080144    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080148    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080152    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080154    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080157    2576 flags.go:64] FLAG: --lock-file=""
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080160    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080162    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080165    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080170    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080173    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080176    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080179    2576 flags.go:64] FLAG: --logging-format="text"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080182    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080185    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080188    2576 flags.go:64] FLAG: --manifest-url=""
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080191    2576 flags.go:64] FLAG: --manifest-url-header=""
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080195    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080198    2576 flags.go:64] FLAG: --max-open-files="1000000"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080203    2576 flags.go:64] FLAG: --max-pods="110"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080206    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080209    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080212    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 15 12:20:10.086354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080215    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080218    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080221    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080224    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080232    2576 flags.go:64] FLAG: --node-status-max-images="50"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080235    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080238    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080241    2576 flags.go:64] FLAG: --pod-cidr=""
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080243    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080248    2576 flags.go:64] FLAG: --pod-manifest-path=""
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080252    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080255    2576 flags.go:64] FLAG: --pods-per-core="0"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080258    2576 flags.go:64] FLAG: --port="10250"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080261    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080264    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02d4d0b778c491274"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080267    2576 flags.go:64] FLAG: --qos-reserved=""
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080271    2576 flags.go:64] FLAG: --read-only-port="10255"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080274    2576 flags.go:64] FLAG: --register-node="true"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080276    2576 flags.go:64] FLAG: --register-with-taints=""
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080280    2576 flags.go:64] FLAG: --registry-burst="10"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080282    2576 flags.go:64] FLAG: --registry-qps="5"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080285    2576 flags.go:64] FLAG: --reserved-cpus=""
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080288    2576 flags.go:64] FLAG: --reserved-memory=""
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080292    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080295    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 15 12:20:10.086952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080297    2576 flags.go:64] FLAG: --rotate-certificates="false"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080300    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080303    2576 flags.go:64] FLAG: --runonce="false"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080306    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080310    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080313    2576 flags.go:64] FLAG: --seccomp-default="false"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080316    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080319    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080322    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080325    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080328    2576 flags.go:64] FLAG: --storage-driver-password="root"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080331    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080334    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080336    2576 flags.go:64] FLAG: --storage-driver-user="root"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080339    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080342    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080345    2576 flags.go:64] FLAG: --system-cgroups=""
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080349    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080354    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080357    2576 flags.go:64] FLAG: --tls-cert-file=""
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080360    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080363    2576 flags.go:64] FLAG: --tls-min-version=""
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080366    2576 flags.go:64] FLAG: --tls-private-key-file=""
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080369    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080372    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 15 12:20:10.087606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080374    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080377    2576 flags.go:64] FLAG: --v="2"
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080381    2576 flags.go:64] FLAG: --version="false"
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080385    2576 flags.go:64] FLAG: --vmodule=""
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080388    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.080391    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080503    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080507    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080510    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080513    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080516    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080520    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080523    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080526    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080529    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080532    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080534    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080537    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080540    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080542    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080545    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080547    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:20:10.088234 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080550    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080552    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080556    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080559    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080562    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080564    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080567    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080570    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080572    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080575    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080577    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080579    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080582    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080584    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080587    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080589    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080592    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080594    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080597    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080599    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:20:10.088783 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080602    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080605    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080608    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080611    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080614    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080616    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080619    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080622    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080624    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080627    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080629    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080632    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080634    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080636    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080640    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080642    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080645    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080649    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080653    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:20:10.089285 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080656    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080659    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080662    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080665    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080667    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080671    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080673    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080676    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080678    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080681    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080683    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080686    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080688    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080690    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080694    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080698    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080702    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080705    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080708    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:20:10.089732 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080711    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080714    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080717    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080720    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080723    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080725    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080728    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080730    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080734    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080736    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080739    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080742    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080744    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080747    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080749    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080753    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080768    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080771    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080774    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080776    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:20:10.090354 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080779    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080781    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080784    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080786    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.080788    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.081307    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.087658    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.087671    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.087687    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.087692    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087755    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087760    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087763    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087766    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087768    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:20:10.090825 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087771    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087774    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087777    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087780    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087783    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087786    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087789    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087791    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087794    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087796    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087799    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087803    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087806    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087808    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087811    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087814    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087816    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087819    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087821    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087824    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:20:10.091206 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087827    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087830    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087833    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087835    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087839    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087843    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087846    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087853    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087856    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087859    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087862    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087864    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087867    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087869    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087872    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087874    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087877    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087880    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087882    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087885    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:20:10.091717 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087887    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087890    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087893    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087895    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087904    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087906    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087909    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087912    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087915    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087919    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087922    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087946    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087953    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087956    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087959    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087962    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087964    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087967    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087969    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:20:10.092216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087972    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087975    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087977    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087980    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087982    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087984    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087987    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087989    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087992    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087994    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087997    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.087999    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088002    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088005    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088008    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088010    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088013    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088016    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088025    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:20:10.092680 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088027    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088030    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088033    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088035    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088038    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088040    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088043    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088045    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088048    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088051    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088053    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088056    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088059    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088061    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088064    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:20:10.093187 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088066    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.088072    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088215    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088220    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088223    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088225    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088230    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088233    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088236    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088239    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088243    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088245    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088248    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088251    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 15 12:20:10.093548 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088254    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088256    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088259    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088267    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088269    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088272    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088274    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088277    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088280    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088282    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088284    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088287    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088289    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088292    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088294    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088297    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088299    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088302    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088304    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088306    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 15 12:20:10.093885 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088309    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088311    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088319    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088322    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088325    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088328    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088330    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088332    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088335    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088338    2576 feature_gate.go:328] unrecognized feature gate: Example2
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088341    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088344    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088346    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088349    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088351    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088354    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088362    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088365    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088367    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088371    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 15 12:20:10.094626 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088374    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088376    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088379    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088381    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088384    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088386    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088389    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088391    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088394    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088396    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088399    2576 feature_gate.go:328] unrecognized feature gate: Example
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088401    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088403    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088406    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088408    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088411    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088413    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088416    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088418    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088420    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 15 12:20:10.095301 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088423    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088426    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088429    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088431    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088434    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088437    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088440    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088442    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088444    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088452    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088455    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088458    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088460    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088462    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088465    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088469    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088472    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088475    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088478    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088481    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 15 12:20:10.095881 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088483    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088486    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088489    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088492    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088494    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088497    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:10.088500    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.088505    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.088610    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.090636    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.091821    2576 server.go:1019] "Starting client certificate rotation"
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.091940    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.091979    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 15 12:20:10.096387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.094192    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 12:20:10.096798 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.095739    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 15 12:20:10.103964 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.103946    2576 log.go:25] "Validated CRI v1 runtime API"
Jul 15 12:20:10.105801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.105788    2576 log.go:25] "Validated CRI v1 image API"
Jul 15 12:20:10.106589 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.106572    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 15 12:20:10.109366 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.109347    2576 fs.go:135] Filesystem UUIDs: map[1c682262-7e6b-4db4-903c-29fb2f2ff059:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8f455455-a8e1-4384-9834-fb64efb950b7:/dev/nvme0n1p4]
Jul 15 12:20:10.109440 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.109364    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 15 12:20:10.114937 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.114816    2576 manager.go:216] Machine: {Timestamp:2026-07-15 12:20:10.113669043 +0000 UTC m=+0.272995080 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3197588 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec273097b166df5d6a9331e5277b652c SystemUUID:ec273097-b166-df5d-6a93-31e5277b652c BootID:d8af7638-a5d1-46dd-827a-1fc21178fd84 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:53:d9:ba:b6:3b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:53:d9:ba:b6:3b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:54:47:10:97:20 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 15 12:20:10.114937 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.114920    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 15 12:20:10.115049 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.115003    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 15 12:20:10.117604 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.117580    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 15 12:20:10.117727 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.117606    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 15 12:20:10.118210 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.118201    2576 topology_manager.go:138] "Creating topology manager with none policy"
Jul 15 12:20:10.118239 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.118213    2576 container_manager_linux.go:309] "Creating device plugin manager"
Jul 15 12:20:10.118239 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.118220    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 12:20:10.118899 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.118889    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 15 12:20:10.119894 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.119883    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 15 12:20:10.123785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.123772    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 12:20:10.123893 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.123883    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 15 12:20:10.124452 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124441    2576 kubelet.go:486] "Attempting to sync node with API server"
Jul 15 12:20:10.124509 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124454    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 15 12:20:10.124509 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124471    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 15 12:20:10.124509 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124480    2576 kubelet.go:398] "Adding apiserver pod source"
Jul 15 12:20:10.124509 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124488    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 15 12:20:10.124904 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124889    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 12:20:10.124974 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.124914    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 15 12:20:10.126746 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.126728    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 15 12:20:10.127369 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.127350    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 12:20:10.128251 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.128230    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 15 12:20:10.128339 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.128274    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 15 12:20:10.129127 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129114    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 15 12:20:10.129197 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129136    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 15 12:20:10.129197 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129147    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 15 12:20:10.129197 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129159    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 15 12:20:10.129197 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129178    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 15 12:20:10.129197 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129189    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129208    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129220    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129236    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129244    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129256    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129264    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129293    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 15 12:20:10.129401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.129301    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 15 12:20:10.132584 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.132567    2576 server.go:1268] "Started kubelet"
Jul 15 12:20:10.132660 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.132633    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 15 12:20:10.132739 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.132701    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 15 12:20:10.132785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.132755    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 15 12:20:10.133482 ip-10-0-135-6 systemd[1]: Started Kubernetes Kubelet.
Jul 15 12:20:10.133819 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.133798    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 15 12:20:10.138183 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.138167    2576 server.go:310] "Adding debug handlers to kubelet server"
Jul 15 12:20:10.138413 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.138393    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 15 12:20:10.139034 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.138996    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 15 12:20:10.139034 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.139010    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 15 12:20:10.139146 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.138370    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-6.ec2.internal.18c275e0a7537b27  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-6.ec2.internal,UID:ip-10-0-135-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-6.ec2.internal,},FirstTimestamp:2026-07-15 12:20:10.132527911 +0000 UTC m=+0.291853949,LastTimestamp:2026-07-15 12:20:10.132527911 +0000 UTC m=+0.291853949,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-6.ec2.internal,}"
Jul 15 12:20:10.139554 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.139525    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 15 12:20:10.140165 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140147    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 15 12:20:10.140165 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140161    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 15 12:20:10.140299 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.140259    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:10.140725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140707    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 15 12:20:10.140725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140712    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 15 12:20:10.140856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140738    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 15 12:20:10.140856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140839    2576 reconstruct.go:97] "Volume reconstruction finished"
Jul 15 12:20:10.140856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.140849    2576 reconciler.go:29] "Reconciler: start to sync state"
Jul 15 12:20:10.141018 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141002    2576 factory.go:55] Registering systemd factory
Jul 15 12:20:10.141018 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141019    2576 factory.go:223] Registration of the systemd container factory successfully
Jul 15 12:20:10.141200 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141183    2576 factory.go:153] Registering CRI-O factory
Jul 15 12:20:10.141200 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141201    2576 factory.go:223] Registration of the crio container factory successfully
Jul 15 12:20:10.141313 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141247    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 15 12:20:10.141313 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141268    2576 factory.go:103] Registering Raw factory
Jul 15 12:20:10.141313 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141284    2576 manager.go:1196] Started watching for new ooms in manager
Jul 15 12:20:10.141575 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.141565    2576 manager.go:349] Starting recovery of all containers
Jul 15 12:20:10.142595 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.142573    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 15 12:20:10.151249 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.151115    2576 manager.go:354] Recovery completed
Jul 15 12:20:10.153172 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.153151    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 15 12:20:10.155392 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.155380    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:10.157270 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157248    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:10.157339 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157283    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:10.157339 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157293    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:10.157837 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157821    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 15 12:20:10.157837 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157831    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 15 12:20:10.157973 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157837    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:10.157973 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.157846    2576 state_mem.go:36] "Initialized new in-memory state store"
Jul 15 12:20:10.158607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.158595    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:10.158643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.158618    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:10.158643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.158627    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:10.159920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.159908    2576 policy_none.go:49] "None policy: Start"
Jul 15 12:20:10.159998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.159972    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 15 12:20:10.159998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.159985    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 15 12:20:10.161882 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.161873    2576 policy_none.go:47] "Start"
Jul 15 12:20:10.168640 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.168543    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-6.ec2.internal.18c275e0a8cd04a5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-6.ec2.internal,UID:ip-10-0-135-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-6.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-6.ec2.internal,},FirstTimestamp:2026-07-15 12:20:10.157270181 +0000 UTC m=+0.316596218,LastTimestamp:2026-07-15 12:20:10.157270181 +0000 UTC m=+0.316596218,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-6.ec2.internal,}"
Jul 15 12:20:10.182069 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.182001    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-6.ec2.internal.18c275e0a8cd4976  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-6.ec2.internal,UID:ip-10-0-135-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-6.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-6.ec2.internal,},FirstTimestamp:2026-07-15 12:20:10.157287798 +0000 UTC m=+0.316613835,LastTimestamp:2026-07-15 12:20:10.157287798 +0000 UTC m=+0.316613835,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-6.ec2.internal,}"
Jul 15 12:20:10.188313 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.188294    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hshr2"
Jul 15 12:20:10.193042 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.192986    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-6.ec2.internal.18c275e0a8cd6eea  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-6.ec2.internal,UID:ip-10-0-135-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-6.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-6.ec2.internal,},FirstTimestamp:2026-07-15 12:20:10.157297386 +0000 UTC m=+0.316623428,LastTimestamp:2026-07-15 12:20:10.157297386 +0000 UTC m=+0.316623428,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-6.ec2.internal,}"
Jul 15 12:20:10.194637 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.194623    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hshr2"
Jul 15 12:20:10.206474 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206462    2576 manager.go:337] "Starting Device Plugin manager"
Jul 15 12:20:10.206528 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.206496    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 15 12:20:10.206528 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206505    2576 server.go:87] "Starting device plugin registration server"
Jul 15 12:20:10.206705 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206696    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 15 12:20:10.206738 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206707    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 15 12:20:10.206840 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206811    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 15 12:20:10.206923 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206884    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 15 12:20:10.206923 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:10.206895    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 15 12:20:10.207495 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.207477    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 15 12:20:10.207574 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:10.207519    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:11.564628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.564600    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 15 12:20:11.565787 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.565769    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 15 12:20:11.565875 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.565801    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 15 12:20:11.565875 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.565819    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 15 12:20:11.565875 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.565851    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 15 12:20:11.568252 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.568234    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:20:11.590113 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.590094    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:11.591272 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.591250    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:11.591363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.591282    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:11.591363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.591295    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:11.591363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.591323    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.601609 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.601592    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.601679 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.601612    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-6.ec2.internal\": node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:11.634790 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.634737    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:11.666347 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.666310    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"]
Jul 15 12:20:11.666453 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.666394    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:11.667299 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.667283    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:11.667382 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.667310    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:11.667382 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.667321    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:11.669711 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.669699    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:11.669846 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.669830    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.669893 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.669862    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:11.670481 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.670467    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:11.670579 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.670493    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:11.670579 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.670504    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:11.670579 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.670547    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:11.670579 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.670571    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:11.670766 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.670585    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:11.672685 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.672670    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.672772 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.672693    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:11.673393 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.673380    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:11.673462 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.673409    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:11.673462 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.673421    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:11.702263 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.702244    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-6.ec2.internal\" not found" node="ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.706781 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.706768    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-6.ec2.internal\" not found" node="ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.725457 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.725441    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ba60970e3daa3fb7b1af72cb6685b19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal\" (UID: \"8ba60970e3daa3fb7b1af72cb6685b19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.725555 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.725465    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba60970e3daa3fb7b1af72cb6685b19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal\" (UID: \"8ba60970e3daa3fb7b1af72cb6685b19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.725555 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.725481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd7265a5e5fed270cfc17e07cb6ae380-config\") pod \"kube-apiserver-proxy-ip-10-0-135-6.ec2.internal\" (UID: \"bd7265a5e5fed270cfc17e07cb6ae380\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.735575 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.735554    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:11.825632 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.825598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ba60970e3daa3fb7b1af72cb6685b19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal\" (UID: \"8ba60970e3daa3fb7b1af72cb6685b19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.825632 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.825638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba60970e3daa3fb7b1af72cb6685b19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal\" (UID: \"8ba60970e3daa3fb7b1af72cb6685b19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.825841 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.825657    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd7265a5e5fed270cfc17e07cb6ae380-config\") pod \"kube-apiserver-proxy-ip-10-0-135-6.ec2.internal\" (UID: \"bd7265a5e5fed270cfc17e07cb6ae380\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.825841 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.825702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd7265a5e5fed270cfc17e07cb6ae380-config\") pod \"kube-apiserver-proxy-ip-10-0-135-6.ec2.internal\" (UID: \"bd7265a5e5fed270cfc17e07cb6ae380\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.825841 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.825714    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba60970e3daa3fb7b1af72cb6685b19-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal\" (UID: \"8ba60970e3daa3fb7b1af72cb6685b19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.825841 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:11.825702    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ba60970e3daa3fb7b1af72cb6685b19-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal\" (UID: \"8ba60970e3daa3fb7b1af72cb6685b19\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:11.835631 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.835608    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:11.936481 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:11.936422    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.006636 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.006606    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:12.009757 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.009740    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:12.037085 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.037046    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.137455 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.137428    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.237955 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.237882    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.338464 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.338436    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.374602 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.374579    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 15 12:20:12.375181 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.374720    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 15 12:20:12.439066 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.439028    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.481854 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.481817    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 12:15:11 +0000 UTC" deadline="2028-01-11 17:00:57.105897034 +0000 UTC"
Jul 15 12:20:12.481854 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.481848    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13084h40m44.624053985s"
Jul 15 12:20:12.539602 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.539511    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.548915 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:12.548886    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba60970e3daa3fb7b1af72cb6685b19.slice/crio-a46ce4b6f77dd15f9b1ccf54a7cd80fddd92f635c4481514ad065bec9fb5afe7 WatchSource:0}: Error finding container a46ce4b6f77dd15f9b1ccf54a7cd80fddd92f635c4481514ad065bec9fb5afe7: Status 404 returned error can't find the container with id a46ce4b6f77dd15f9b1ccf54a7cd80fddd92f635c4481514ad065bec9fb5afe7
Jul 15 12:20:12.549200 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:12.549181    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7265a5e5fed270cfc17e07cb6ae380.slice/crio-37819d92d97b23d963f2ef21e8519c8ba55e6f324eeaf78ce98fc2bf5d4be7ff WatchSource:0}: Error finding container 37819d92d97b23d963f2ef21e8519c8ba55e6f324eeaf78ce98fc2bf5d4be7ff: Status 404 returned error can't find the container with id 37819d92d97b23d963f2ef21e8519c8ba55e6f324eeaf78ce98fc2bf5d4be7ff
Jul 15 12:20:12.553623 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.553609    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:20:12.568864 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.568826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal" event={"ID":"bd7265a5e5fed270cfc17e07cb6ae380","Type":"ContainerStarted","Data":"37819d92d97b23d963f2ef21e8519c8ba55e6f324eeaf78ce98fc2bf5d4be7ff"}
Jul 15 12:20:12.569864 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.569844    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal" event={"ID":"8ba60970e3daa3fb7b1af72cb6685b19","Type":"ContainerStarted","Data":"a46ce4b6f77dd15f9b1ccf54a7cd80fddd92f635c4481514ad065bec9fb5afe7"}
Jul 15 12:20:12.586564 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.586540    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 15 12:20:12.587311 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.587292    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientMemory"
Jul 15 12:20:12.587377 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.587322    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasNoDiskPressure"
Jul 15 12:20:12.587377 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.587333    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeHasSufficientPID"
Jul 15 12:20:12.625289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.625267    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:20:12.639949 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.639930    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.740474 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.740446    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.841004 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.840928    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.941761 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:12.941728    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-6.ec2.internal\" not found"
Jul 15 12:20:12.946590 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:12.946566    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:20:13.021401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.021372    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:20:13.023428 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.023405    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:13.039443 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.039421    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 12:20:13.040434 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.040416    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal"
Jul 15 12:20:13.050454 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.050434    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 15 12:20:13.348097 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.348073    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 15 12:20:13.408154 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.408003    2576 apiserver.go:52] "Watching apiserver"
Jul 15 12:20:13.419611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.419586    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 15 12:20:13.419968 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.419940    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-2jtcw","openshift-dns/node-resolver-hgvm9","openshift-image-registry/node-ca-fjhkd","openshift-multus/multus-additional-cni-plugins-kkfz5","openshift-multus/multus-spzrf","kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal","openshift-multus/network-metrics-daemon-zjzm8","openshift-network-diagnostics/network-check-target-mbqqd","openshift-network-operator/iptables-alerter-9brfz","openshift-ovn-kubernetes/ovnkube-node-8fhzw","kube-system/global-pull-secret-syncer-7kfpt","kube-system/konnectivity-agent-wswpt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"]
Jul 15 12:20:13.422936 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.422571    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.422936 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.422886    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 15 12:20:13.425206 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.425178    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.426526 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.426355    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.426758 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.426740    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-smkjm\""
Jul 15 12:20:13.427131 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.427113    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 15 12:20:13.427291 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.427249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.428922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.428359    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4cm72\""
Jul 15 12:20:13.428922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.428572    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.428922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.428851    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 15 12:20:13.430229 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.429633    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 15 12:20:13.430229 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.429647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.430229 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.429836    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.430229 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.430055    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 15 12:20:13.430481 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.430345    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.430553 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.430489    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-82g49\""
Jul 15 12:20:13.432201 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.431983    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.434356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.434356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434116    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 15 12:20:13.434723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77-agent-certs\") pod \"konnectivity-agent-wswpt\" (UID: \"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77\") " pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.434723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434551    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77-konnectivity-ca\") pod \"konnectivity-agent-wswpt\" (UID: \"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77\") " pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.434723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/678138d5-cba2-4e9f-b8b0-a492d7eca657-iptables-alerter-script\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.434723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434602    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/678138d5-cba2-4e9f-b8b0-a492d7eca657-host-slash\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.434723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434627    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw89k\" (UniqueName: \"kubernetes.io/projected/678138d5-cba2-4e9f-b8b0-a492d7eca657-kube-api-access-sw89k\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.435045 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.434762    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 15 12:20:13.435045 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.435019    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-856tb\""
Jul 15 12:20:13.435801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.435778    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 15 12:20:13.435887 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.435859    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 15 12:20:13.436296 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.436273    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6llfh\""
Jul 15 12:20:13.436369 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.436306    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.436450 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.436434    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7sqrn\""
Jul 15 12:20:13.436747 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.436732    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:13.436835 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.436809    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:13.436988 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.436964    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 15 12:20:13.437057 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.436967    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.437461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.437444    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.437553 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.437514    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.438776 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.438757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:13.438871 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.438850    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:13.440977 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.440957    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.441193 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.441176    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 15 12:20:13.443559 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.443526    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wrdf9\""
Jul 15 12:20:13.443645 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.443586    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.443854 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.443833    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.444114 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.444097    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.446000 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.445982    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.446122 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.446102    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.446699 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.446461    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.446797 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.446713    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hldt9\""
Jul 15 12:20:13.448999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.448981    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-b2zh5\""
Jul 15 12:20:13.449325 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.449308    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 15 12:20:13.449954 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.449698    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.449954 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.449762    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:13.449954 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.449868    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 15 12:20:13.452060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.451121    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 15 12:20:13.452060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.451306    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 15 12:20:13.452060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.451616    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 15 12:20:13.452060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.451942    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 15 12:20:13.482178 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.482160    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hp55s"
Jul 15 12:20:13.495221 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.495198    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hp55s"
Jul 15 12:20:13.525470 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.525410    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 15 12:20:13.535009 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.534984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysctl-conf\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.535104 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535024    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-run-netns\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535104 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535051    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-systemd\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535104 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535104 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535097    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535122    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovnkube-config\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/678138d5-cba2-4e9f-b8b0-a492d7eca657-iptables-alerter-script\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535177    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535203    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysconfig\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535224    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-kubernetes\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-env-overrides\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535276    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15a5102c-c886-4f37-ba0c-2bb0f653fa09-serviceca\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-cni-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.535372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77-agent-certs\") pod \"konnectivity-agent-wswpt\" (UID: \"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77\") " pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d6d2951-2153-434e-b87f-bff9e54fb088-tmp-dir\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535412    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5drt\" (UniqueName: \"kubernetes.io/projected/3d6d2951-2153-434e-b87f-bff9e54fb088-kube-api-access-c5drt\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-cni-netd\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovnkube-script-lib\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-sys\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-kubelet\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535560    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-conf-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535584    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-etc-kubernetes\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535606    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxm5d\" (UniqueName: \"kubernetes.io/projected/170d18be-bfdd-4b12-8f22-62242c6f6430-kube-api-access-bxm5d\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535629    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-device-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv2dw\" (UniqueName: \"kubernetes.io/projected/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-kube-api-access-gv2dw\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d6d2951-2153-434e-b87f-bff9e54fb088-hosts-file\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535711    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-systemd\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535735    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-kubelet\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfbbt\" (UniqueName: \"kubernetes.io/projected/c3e01600-0295-4781-b1e1-5825eb79bad8-kube-api-access-sfbbt\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535809    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-cni-multus\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.535843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535832    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-hostroot\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535853    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-multus-certs\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzwm6\" (UniqueName: \"kubernetes.io/projected/877ad258-985f-4f41-ba9b-df2d9e3b4133-kube-api-access-mzwm6\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535903    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-run-ovn-kubernetes\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovn-node-metrics-cert\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535950    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-system-cni-dir\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535976    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-socket-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.535999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15a5102c-c886-4f37-ba0c-2bb0f653fa09-host\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-os-release\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536070    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-system-cni-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536092    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-cnibin\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-daemon-config\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536143    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-var-lib-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-ovn\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536198    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-log-socket\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.536624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536225    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-socket-dir-parent\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536250    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-sys-fs\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3334fedd-620f-40db-9aed-a13da54e73b2-etc-tuned\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk6f6\" (UniqueName: \"kubernetes.io/projected/15a5102c-c886-4f37-ba0c-2bb0f653fa09-kube-api-access-dk6f6\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536325    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/678138d5-cba2-4e9f-b8b0-a492d7eca657-host-slash\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw89k\" (UniqueName: \"kubernetes.io/projected/678138d5-cba2-4e9f-b8b0-a492d7eca657-kube-api-access-sw89k\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-var-lib-kubelet\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536426    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-systemd-units\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-slash\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536486    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536512    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-netns\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-lib-modules\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536577    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-cni-bin\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536599    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/170d18be-bfdd-4b12-8f22-62242c6f6430-cni-binary-copy\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.537340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536648    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysctl-d\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-etc-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-cni-bin\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536722    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536753    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-node-log\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536779    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-dbus\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536803    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-k8s-cni-cncf-io\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-run\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536849    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3334fedd-620f-40db-9aed-a13da54e73b2-tmp\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536873    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzp9t\" (UniqueName: \"kubernetes.io/projected/3334fedd-620f-40db-9aed-a13da54e73b2-kube-api-access-wzp9t\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmvkj\" (UniqueName: \"kubernetes.io/projected/c87a0dff-b6af-428d-b7fc-0091fa66d911-kube-api-access-nmvkj\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-kubelet-config\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.536999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-os-release\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537025    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-registration-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537079    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-host\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537104    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-cnibin\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.538060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537133    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.538853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.538853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537201    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77-konnectivity-ca\") pod \"konnectivity-agent-wswpt\" (UID: \"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77\") " pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.538853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537228    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-modprobe-d\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.538853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.537878    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/678138d5-cba2-4e9f-b8b0-a492d7eca657-iptables-alerter-script\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.538853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.538301    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 15 12:20:13.539089 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.539043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/678138d5-cba2-4e9f-b8b0-a492d7eca657-host-slash\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.540620 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.540597    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77-konnectivity-ca\") pod \"konnectivity-agent-wswpt\" (UID: \"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77\") " pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.542048 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.542026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77-agent-certs\") pod \"konnectivity-agent-wswpt\" (UID: \"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77\") " pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.548283 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.548252    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw89k\" (UniqueName: \"kubernetes.io/projected/678138d5-cba2-4e9f-b8b0-a492d7eca657-kube-api-access-sw89k\") pod \"iptables-alerter-9brfz\" (UID: \"678138d5-cba2-4e9f-b8b0-a492d7eca657\") " pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.638368 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638290    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-node-log\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.638368 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638326    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-dbus\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.638368 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638352    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-k8s-cni-cncf-io\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638377    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-run\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638402    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3334fedd-620f-40db-9aed-a13da54e73b2-tmp\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638423    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzp9t\" (UniqueName: \"kubernetes.io/projected/3334fedd-620f-40db-9aed-a13da54e73b2-kube-api-access-wzp9t\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638427    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-node-log\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638456    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-run\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638420    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-k8s-cni-cncf-io\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmvkj\" (UniqueName: \"kubernetes.io/projected/c87a0dff-b6af-428d-b7fc-0091fa66d911-kube-api-access-nmvkj\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-kubelet-config\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-os-release\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-registration-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638607    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-host\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638628    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-cnibin\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-dbus\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.638655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-registration-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-kubelet-config\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638703    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-os-release\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638727    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-host\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-modprobe-d\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysctl-conf\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638958    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-modprobe-d\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.638985    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-run-netns\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639020    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-run-netns\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysctl-conf\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-systemd\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639086    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-systemd\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639138    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.639308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovnkube-config\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639157    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-cnibin\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639210    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysconfig\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639262    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysconfig\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-kubernetes\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639292    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.639299    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639307    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-env-overrides\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15a5102c-c886-4f37-ba0c-2bb0f653fa09-serviceca\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639348    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-kubernetes\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.639406    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:14.139348262 +0000 UTC m=+3.016071248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639425    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639451    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-cni-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639479    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d6d2951-2153-434e-b87f-bff9e54fb088-tmp-dir\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639504    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5drt\" (UniqueName: \"kubernetes.io/projected/3d6d2951-2153-434e-b87f-bff9e54fb088-kube-api-access-c5drt\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.640169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-cni-netd\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovnkube-script-lib\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-sys\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-kubelet\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-conf-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-etc-kubernetes\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639735    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxm5d\" (UniqueName: \"kubernetes.io/projected/170d18be-bfdd-4b12-8f22-62242c6f6430-kube-api-access-bxm5d\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639757    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-device-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv2dw\" (UniqueName: \"kubernetes.io/projected/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-kube-api-access-gv2dw\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d6d2951-2153-434e-b87f-bff9e54fb088-hosts-file\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-systemd\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-kubelet\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfbbt\" (UniqueName: \"kubernetes.io/projected/c3e01600-0295-4781-b1e1-5825eb79bad8-kube-api-access-sfbbt\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-cni-multus\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-hostroot\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639938    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-multus-certs\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d6d2951-2153-434e-b87f-bff9e54fb088-tmp-dir\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.640959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzwm6\" (UniqueName: \"kubernetes.io/projected/877ad258-985f-4f41-ba9b-df2d9e3b4133-kube-api-access-mzwm6\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.639989    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-run-ovn-kubernetes\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640031    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovn-node-metrics-cert\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640078    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-cni-netd\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640064    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-system-cni-dir\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640122    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-socket-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovnkube-config\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15a5102c-c886-4f37-ba0c-2bb0f653fa09-host\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-etc-selinux\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-os-release\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640245    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-system-cni-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640270    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-cni-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-cni-binary-copy\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640322    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-os-release\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640320    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640378    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3d6d2951-2153-434e-b87f-bff9e54fb088-hosts-file\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.641814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640383    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-conf-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640382    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-sys\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640405    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-kubelet\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640445    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-systemd\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-device-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15a5102c-c886-4f37-ba0c-2bb0f653fa09-host\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-cni-multus\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-system-cni-dir\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovnkube-script-lib\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640794    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-etc-kubernetes\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640997    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15a5102c-c886-4f37-ba0c-2bb0f653fa09-serviceca\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641198    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c3e01600-0295-4781-b1e1-5825eb79bad8-system-cni-dir\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-run-ovn-kubernetes\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641241    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c87a0dff-b6af-428d-b7fc-0091fa66d911-env-overrides\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-multus-certs\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641370    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-hostroot\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.640271    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-cnibin\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641421    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-daemon-config\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.642630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641474    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-var-lib-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-socket-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641500    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-ovn\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-log-socket\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641564    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-run-ovn\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-socket-dir-parent\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641631    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-sys-fs\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3334fedd-620f-40db-9aed-a13da54e73b2-etc-tuned\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-cnibin\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641691    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk6f6\" (UniqueName: \"kubernetes.io/projected/15a5102c-c886-4f37-ba0c-2bb0f653fa09-kube-api-access-dk6f6\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641735    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-var-lib-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-var-lib-kubelet\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-systemd-units\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641804    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-slash\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641828    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641857    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-netns\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.643582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-lib-modules\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-cni-bin\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641940    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/170d18be-bfdd-4b12-8f22-62242c6f6430-cni-binary-copy\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.641995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysctl-d\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642025    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-etc-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-cni-bin\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642096    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-socket-dir-parent\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642224    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-var-lib-kubelet\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642273    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-systemd-units\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-slash\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642422    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-run-netns\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642467    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-etc-sysctl-d\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642516    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3334fedd-620f-40db-9aed-a13da54e73b2-lib-modules\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642554    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-sys-fs\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-cni-bin\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.642026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-log-socket\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.644349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c3e01600-0295-4781-b1e1-5825eb79bad8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643060    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/170d18be-bfdd-4b12-8f22-62242c6f6430-cni-binary-copy\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-etc-openvswitch\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/170d18be-bfdd-4b12-8f22-62242c6f6430-host-var-lib-cni-bin\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643209    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c87a0dff-b6af-428d-b7fc-0091fa66d911-host-kubelet\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.643279    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.643344    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:14.143326476 +0000 UTC m=+3.020049464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643633    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3334fedd-620f-40db-9aed-a13da54e73b2-tmp\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.645170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.643691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/170d18be-bfdd-4b12-8f22-62242c6f6430-multus-daemon-config\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.645976 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.645953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c87a0dff-b6af-428d-b7fc-0091fa66d911-ovn-node-metrics-cert\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.647806 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.647781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3334fedd-620f-40db-9aed-a13da54e73b2-etc-tuned\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.660469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.660441    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmvkj\" (UniqueName: \"kubernetes.io/projected/c87a0dff-b6af-428d-b7fc-0091fa66d911-kube-api-access-nmvkj\") pod \"ovnkube-node-8fhzw\" (UID: \"c87a0dff-b6af-428d-b7fc-0091fa66d911\") " pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:13.661098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.661065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzp9t\" (UniqueName: \"kubernetes.io/projected/3334fedd-620f-40db-9aed-a13da54e73b2-kube-api-access-wzp9t\") pod \"tuned-2jtcw\" (UID: \"3334fedd-620f-40db-9aed-a13da54e73b2\") " pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.670437 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.670414    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:13.670521 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.670447    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:13.670521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.670416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk6f6\" (UniqueName: \"kubernetes.io/projected/15a5102c-c886-4f37-ba0c-2bb0f653fa09-kube-api-access-dk6f6\") pod \"node-ca-fjhkd\" (UID: \"15a5102c-c886-4f37-ba0c-2bb0f653fa09\") " pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.670521 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.670462    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:13.670706 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:13.670579    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:14.170557951 +0000 UTC m=+3.047280950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:13.672064 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.672041    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5drt\" (UniqueName: \"kubernetes.io/projected/3d6d2951-2153-434e-b87f-bff9e54fb088-kube-api-access-c5drt\") pod \"node-resolver-hgvm9\" (UID: \"3d6d2951-2153-434e-b87f-bff9e54fb088\") " pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.672359 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.672332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv2dw\" (UniqueName: \"kubernetes.io/projected/c6a26546-f70d-4359-b5e9-2cf155a7a2b9-kube-api-access-gv2dw\") pod \"aws-ebs-csi-driver-node-jzb89\" (UID: \"c6a26546-f70d-4359-b5e9-2cf155a7a2b9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.672735 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.672716    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzwm6\" (UniqueName: \"kubernetes.io/projected/877ad258-985f-4f41-ba9b-df2d9e3b4133-kube-api-access-mzwm6\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:13.673791 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.673772    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfbbt\" (UniqueName: \"kubernetes.io/projected/c3e01600-0295-4781-b1e1-5825eb79bad8-kube-api-access-sfbbt\") pod \"multus-additional-cni-plugins-kkfz5\" (UID: \"c3e01600-0295-4781-b1e1-5825eb79bad8\") " pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.680029 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.680003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxm5d\" (UniqueName: \"kubernetes.io/projected/170d18be-bfdd-4b12-8f22-62242c6f6430-kube-api-access-bxm5d\") pod \"multus-spzrf\" (UID: \"170d18be-bfdd-4b12-8f22-62242c6f6430\") " pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.735842 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.735819    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9brfz"
Jul 15 12:20:13.743397 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.743375    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:13.757054 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.757039    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fjhkd"
Jul 15 12:20:13.761990 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.761974    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kkfz5"
Jul 15 12:20:13.769543 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.769507    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-spzrf"
Jul 15 12:20:13.777062 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.777045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89"
Jul 15 12:20:13.783634 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.783616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hgvm9"
Jul 15 12:20:13.791114 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.791097    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2jtcw"
Jul 15 12:20:13.797681 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:13.797662    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:14.145731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.145697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:14.145904 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.145771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:14.145904 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.145884    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:14.145993 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.145964    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:15.145944437 +0000 UTC m=+4.022667431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:14.145993 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.145889    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:14.146079 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.146030    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:15.146016057 +0000 UTC m=+4.022739045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:14.246787 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.246749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:14.246928 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.246910    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:14.246990 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.246934    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:14.246990 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.246944    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:14.247066 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.246997    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:15.246981408 +0000 UTC m=+4.123704399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:14.348032 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.347822    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a26546_f70d_4359_b5e9_2cf155a7a2b9.slice/crio-9d09d43c303cd07a5fe91f0283257effb1ec99e0f335e0c0af333321d03790c3 WatchSource:0}: Error finding container 9d09d43c303cd07a5fe91f0283257effb1ec99e0f335e0c0af333321d03790c3: Status 404 returned error can't find the container with id 9d09d43c303cd07a5fe91f0283257effb1ec99e0f335e0c0af333321d03790c3
Jul 15 12:20:14.349929 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.349844    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc87a0dff_b6af_428d_b7fc_0091fa66d911.slice/crio-3ed252e4f00dc8187ca23a60e1d8cd2e73aec361069c3d6bc443bd18f1c62a49 WatchSource:0}: Error finding container 3ed252e4f00dc8187ca23a60e1d8cd2e73aec361069c3d6bc443bd18f1c62a49: Status 404 returned error can't find the container with id 3ed252e4f00dc8187ca23a60e1d8cd2e73aec361069c3d6bc443bd18f1c62a49
Jul 15 12:20:14.350492 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.350469    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e01600_0295_4781_b1e1_5825eb79bad8.slice/crio-ad48bc9560e51880c9aee5c4fc5d2683d9c283a98072ba636c564a64b2383c4f WatchSource:0}: Error finding container ad48bc9560e51880c9aee5c4fc5d2683d9c283a98072ba636c564a64b2383c4f: Status 404 returned error can't find the container with id ad48bc9560e51880c9aee5c4fc5d2683d9c283a98072ba636c564a64b2383c4f
Jul 15 12:20:14.351281 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.351233    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15a5102c_c886_4f37_ba0c_2bb0f653fa09.slice/crio-025d4a90e0586acf76263399520a8229e3cd7195889cd534bc43aeea03eab9e3 WatchSource:0}: Error finding container 025d4a90e0586acf76263399520a8229e3cd7195889cd534bc43aeea03eab9e3: Status 404 returned error can't find the container with id 025d4a90e0586acf76263399520a8229e3cd7195889cd534bc43aeea03eab9e3
Jul 15 12:20:14.374048 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.374026    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3334fedd_620f_40db_9aed_a13da54e73b2.slice/crio-000931a4beb62f7daef42362d792eb220156bade8bb72c83f55adf25558e5c97 WatchSource:0}: Error finding container 000931a4beb62f7daef42362d792eb220156bade8bb72c83f55adf25558e5c97: Status 404 returned error can't find the container with id 000931a4beb62f7daef42362d792eb220156bade8bb72c83f55adf25558e5c97
Jul 15 12:20:14.375566 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.375525    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b02fa8_a98a_4ec5_9e7d_98f2edfc0d77.slice/crio-4d51c926a9c6476745f5f22df275f42ff6ac34c637734d2beb8462abc362b64d WatchSource:0}: Error finding container 4d51c926a9c6476745f5f22df275f42ff6ac34c637734d2beb8462abc362b64d: Status 404 returned error can't find the container with id 4d51c926a9c6476745f5f22df275f42ff6ac34c637734d2beb8462abc362b64d
Jul 15 12:20:14.376437 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.376414    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d6d2951_2153_434e_b87f_bff9e54fb088.slice/crio-def06375cce2802583014d40f857b9facb3549cce329df4d67ad56c343861e4a WatchSource:0}: Error finding container def06375cce2802583014d40f857b9facb3549cce329df4d67ad56c343861e4a: Status 404 returned error can't find the container with id def06375cce2802583014d40f857b9facb3549cce329df4d67ad56c343861e4a
Jul 15 12:20:14.377225 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.377196    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170d18be_bfdd_4b12_8f22_62242c6f6430.slice/crio-0f7c01b9567c79ca8d0145bfeec0cc188519141cc4d55998eaad77dba4e92641 WatchSource:0}: Error finding container 0f7c01b9567c79ca8d0145bfeec0cc188519141cc4d55998eaad77dba4e92641: Status 404 returned error can't find the container with id 0f7c01b9567c79ca8d0145bfeec0cc188519141cc4d55998eaad77dba4e92641
Jul 15 12:20:14.377939 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:14.377859    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678138d5_cba2_4e9f_b8b0_a492d7eca657.slice/crio-8a8b0d9dc12979c81a38cf8420648e2087c1467dad52d0f21dce3a74478b047d WatchSource:0}: Error finding container 8a8b0d9dc12979c81a38cf8420648e2087c1467dad52d0f21dce3a74478b047d: Status 404 returned error can't find the container with id 8a8b0d9dc12979c81a38cf8420648e2087c1467dad52d0f21dce3a74478b047d
Jul 15 12:20:14.498846 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.498811    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:15:13 +0000 UTC" deadline="2027-12-21 08:19:21.674850162 +0000 UTC"
Jul 15 12:20:14.498846 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.498845    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12571h59m7.176013131s"
Jul 15 12:20:14.566952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.566922    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:14.567067 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:14.567029    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:14.573815 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.573789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerStarted","Data":"ad48bc9560e51880c9aee5c4fc5d2683d9c283a98072ba636c564a64b2383c4f"}
Jul 15 12:20:14.574874 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.574854    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89" event={"ID":"c6a26546-f70d-4359-b5e9-2cf155a7a2b9","Type":"ContainerStarted","Data":"9d09d43c303cd07a5fe91f0283257effb1ec99e0f335e0c0af333321d03790c3"}
Jul 15 12:20:14.576791 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.576772    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal" event={"ID":"bd7265a5e5fed270cfc17e07cb6ae380","Type":"ContainerStarted","Data":"8076ffc587a65406e4c69c87ef386e76624c61c69bc93d70b02d612f287447fe"}
Jul 15 12:20:14.579034 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.579009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-spzrf" event={"ID":"170d18be-bfdd-4b12-8f22-62242c6f6430","Type":"ContainerStarted","Data":"0f7c01b9567c79ca8d0145bfeec0cc188519141cc4d55998eaad77dba4e92641"}
Jul 15 12:20:14.579917 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.579900    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wswpt" event={"ID":"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77","Type":"ContainerStarted","Data":"4d51c926a9c6476745f5f22df275f42ff6ac34c637734d2beb8462abc362b64d"}
Jul 15 12:20:14.580981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.580956    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2jtcw" event={"ID":"3334fedd-620f-40db-9aed-a13da54e73b2","Type":"ContainerStarted","Data":"000931a4beb62f7daef42362d792eb220156bade8bb72c83f55adf25558e5c97"}
Jul 15 12:20:14.581897 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.581878    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"3ed252e4f00dc8187ca23a60e1d8cd2e73aec361069c3d6bc443bd18f1c62a49"}
Jul 15 12:20:14.582949 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.582917    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9brfz" event={"ID":"678138d5-cba2-4e9f-b8b0-a492d7eca657","Type":"ContainerStarted","Data":"8a8b0d9dc12979c81a38cf8420648e2087c1467dad52d0f21dce3a74478b047d"}
Jul 15 12:20:14.584006 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.583989    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hgvm9" event={"ID":"3d6d2951-2153-434e-b87f-bff9e54fb088","Type":"ContainerStarted","Data":"def06375cce2802583014d40f857b9facb3549cce329df4d67ad56c343861e4a"}
Jul 15 12:20:14.584956 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.584938    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fjhkd" event={"ID":"15a5102c-c886-4f37-ba0c-2bb0f653fa09","Type":"ContainerStarted","Data":"025d4a90e0586acf76263399520a8229e3cd7195889cd534bc43aeea03eab9e3"}
Jul 15 12:20:14.592580 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:14.592520    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-6.ec2.internal" podStartSLOduration=1.5925095900000001 podStartE2EDuration="1.59250959s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:14.592442116 +0000 UTC m=+3.469165122" watchObservedRunningTime="2026-07-15 12:20:14.59250959 +0000 UTC m=+3.469232596"
Jul 15 12:20:15.155524 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.154737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:15.155524 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.154824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:15.155524 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.154960    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:15.155524 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.155021    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:17.155003071 +0000 UTC m=+6.031726060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:15.155524 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.155436    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:15.155524 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.155487    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:17.155471116 +0000 UTC m=+6.032194106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:15.256785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.256097    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:15.256785 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.256328    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:15.256785 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.256346    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:15.256785 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.256358    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:15.256785 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.256417    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:17.256398666 +0000 UTC m=+6.133121662 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:15.500031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.499918    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 12:15:13 +0000 UTC" deadline="2027-12-10 16:18:22.771689907 +0000 UTC"
Jul 15 12:20:15.500031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.499982    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12315h58m7.271711528s"
Jul 15 12:20:15.567474 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.566900    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:15.567474 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.567039    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:15.567964 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.567814    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:15.567964 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:15.567917    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:15.593215 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.593184    2576 generic.go:356] "Generic (PLEG): container finished" podID="8ba60970e3daa3fb7b1af72cb6685b19" containerID="b511879f6afc459fdbc7c38377592bb2cb7db1b4e50ac3268a9092d6ce46cf98" exitCode=0
Jul 15 12:20:15.594101 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:15.594076    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal" event={"ID":"8ba60970e3daa3fb7b1af72cb6685b19","Type":"ContainerDied","Data":"b511879f6afc459fdbc7c38377592bb2cb7db1b4e50ac3268a9092d6ce46cf98"}
Jul 15 12:20:16.566410 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:16.566373    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:16.566974 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:16.566508    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:16.642271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:16.642231    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal" event={"ID":"8ba60970e3daa3fb7b1af72cb6685b19","Type":"ContainerStarted","Data":"92193661af9095f69fa7e638cf8a7e7a177198502ac15b7d5fda08dab53b1ce7"}
Jul 15 12:20:16.661312 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:16.661258    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-6.ec2.internal" podStartSLOduration=3.661235744 podStartE2EDuration="3.661235744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:16.6612348 +0000 UTC m=+5.537957809" watchObservedRunningTime="2026-07-15 12:20:16.661235744 +0000 UTC m=+5.537958748"
Jul 15 12:20:17.175470 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:17.174779    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:17.175470 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:17.174853    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:17.175470 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.174977    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:17.175470 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.175039    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:21.175020889 +0000 UTC m=+10.051743881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:17.175470 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.175105    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:17.175470 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.175139    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:21.175126192 +0000 UTC m=+10.051849179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:17.276624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:17.276001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:17.276624 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.276162    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:17.276624 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.276182    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:17.276624 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.276194    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:17.276624 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.276249    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:21.276232349 +0000 UTC m=+10.152955337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:17.567256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:17.566713    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:17.567256 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.566860    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:17.567256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:17.566882    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:17.567256 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:17.566992    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:18.566686 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:18.566653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:18.566864 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:18.566789    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:19.566073 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:19.566040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:19.566522 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:19.566040    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:19.566522 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:19.566217    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:19.566849 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:19.566727    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:20.566213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:20.566035    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:20.566213 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:20.566173    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:21.207211 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:21.206982    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:21.207211 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:21.207063    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:21.207211 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.207116    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:21.207211 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.207191    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:29.207169249 +0000 UTC m=+18.083892240 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:21.207211 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.207190    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:21.207603 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.207249    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:29.207235344 +0000 UTC m=+18.083958329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:21.308872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:21.308191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:21.308872 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.308392    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:21.308872 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.308416    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:21.308872 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.308428    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:21.308872 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.308494    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:29.30846778 +0000 UTC m=+18.185190784 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:21.567734 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:21.567623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:21.568144 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.567745    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:21.568144 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:21.568108    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:21.568241 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:21.568198    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:22.566279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:22.566183    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:22.566443 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:22.566334    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:23.566125 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:23.565973    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:23.566125 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:23.566112    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:23.566623 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:23.566152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:23.566623 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:23.566295    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:24.566926 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:24.566891    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:24.567281 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:24.566990    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:25.566962 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:25.566937    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:25.567392 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:25.566938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:25.567392 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:25.567042    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:25.567392 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:25.567154    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:26.566562 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:26.566515    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:26.566755 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:26.566659    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:27.566686 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:27.566653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:27.567127 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:27.566653    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:27.567127 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:27.566804    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:27.567127 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:27.566861    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:28.566483 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:28.566451    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:28.566640 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:28.566559    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:29.267635 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:29.267596    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:29.268059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:29.267675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:29.268059 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.267761    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:29.268059 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.267788    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:29.268059 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.267826    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:45.267807503 +0000 UTC m=+34.144530487 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:29.268059 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.267841    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:45.267835277 +0000 UTC m=+34.144558262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:29.368115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:29.368083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:29.368269 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.368239    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:29.368269 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.368257    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:29.368269 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.368267    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:29.368372 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.368321    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:45.368305769 +0000 UTC m=+34.245028763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:29.566299 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:29.566219    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:29.566447 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.566346    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:29.566447 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:29.566380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:29.566630 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:29.566448    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:30.566360 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:30.566332    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:30.566761 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:30.566434    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:31.567002 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:31.566970    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:31.567448 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:31.567086    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:31.567448 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:31.567170    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:31.567448 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:31.567284    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:32.566308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:32.566265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:32.566472 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:32.566388    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:33.566356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.566091    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:33.567110 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.566155    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:33.567110 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:33.566464    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:33.567110 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:33.566592    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:33.677730 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.677692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hgvm9" event={"ID":"3d6d2951-2153-434e-b87f-bff9e54fb088","Type":"ContainerStarted","Data":"de5665d671873dfbf9445bcae7266988e09a5da60533489f7ed03f012d813463"}
Jul 15 12:20:33.679346 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.679320    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fjhkd" event={"ID":"15a5102c-c886-4f37-ba0c-2bb0f653fa09","Type":"ContainerStarted","Data":"0d45aa8b0551af693b745b1ca6f58213a521eb5c5c139000a1f99ed73ca06fcc"}
Jul 15 12:20:33.681081 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.681056    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e01600-0295-4781-b1e1-5825eb79bad8" containerID="47b32a6448574aeda21d98e51a5c4a7b9b73bc0add1d25c0164ee78b5807acf0" exitCode=0
Jul 15 12:20:33.681163 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.681113    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerDied","Data":"47b32a6448574aeda21d98e51a5c4a7b9b73bc0add1d25c0164ee78b5807acf0"}
Jul 15 12:20:33.682697 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.682679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89" event={"ID":"c6a26546-f70d-4359-b5e9-2cf155a7a2b9","Type":"ContainerStarted","Data":"6db776c933252c3dfbe65760191c1125fcaa7b6d5366d7c883b3c3faa92613c7"}
Jul 15 12:20:33.684089 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.684064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-spzrf" event={"ID":"170d18be-bfdd-4b12-8f22-62242c6f6430","Type":"ContainerStarted","Data":"908b2e6e1b2e72b10302952af9b2899d1ddc13a7afa2c2b28db402279b3c0746"}
Jul 15 12:20:33.685378 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.685360    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wswpt" event={"ID":"33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77","Type":"ContainerStarted","Data":"b2f81c24dbc28ea40d55d1c11215d201d6221be549512cbcb839ff03ae3d1d46"}
Jul 15 12:20:33.686725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.686704    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2jtcw" event={"ID":"3334fedd-620f-40db-9aed-a13da54e73b2","Type":"ContainerStarted","Data":"12bfaa74e1385b941221093bc47da9ef9ebec82ade35b3e49afa177051417408"}
Jul 15 12:20:33.697154 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.697082    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hgvm9" podStartSLOduration=4.16126532 podStartE2EDuration="22.697069942s" totalImagesPullingTime="18.535804622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:33.696682649 +0000 UTC m=+22.573405656" watchObservedRunningTime="2026-07-15 12:20:33.697069942 +0000 UTC m=+22.573792947"
Jul 15 12:20:33.743338 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.743295    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-spzrf" podStartSLOduration=4.202042965 podStartE2EDuration="22.743281467s" totalImagesPullingTime="18.541238502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:33.742785843 +0000 UTC m=+22.619508854" watchObservedRunningTime="2026-07-15 12:20:33.743281467 +0000 UTC m=+22.620004474"
Jul 15 12:20:33.762134 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.762088    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2jtcw" podStartSLOduration=4.224682373 podStartE2EDuration="22.762070923s" totalImagesPullingTime="18.53738855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:33.761672604 +0000 UTC m=+22.638395613" watchObservedRunningTime="2026-07-15 12:20:33.762070923 +0000 UTC m=+22.638793929"
Jul 15 12:20:33.780133 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.780094    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wswpt" podStartSLOduration=4.323398076 podStartE2EDuration="22.780083828s" totalImagesPullingTime="18.456685752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:33.780028196 +0000 UTC m=+22.656751201" watchObservedRunningTime="2026-07-15 12:20:33.780083828 +0000 UTC m=+22.656806832"
Jul 15 12:20:33.812776 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:33.812583    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fjhkd" podStartSLOduration=4.269577855 podStartE2EDuration="22.812571248s" totalImagesPullingTime="18.542993393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:33.799608581 +0000 UTC m=+22.676331588" watchObservedRunningTime="2026-07-15 12:20:33.812571248 +0000 UTC m=+22.689294253"
Jul 15 12:20:34.010455 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.010411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:34.011106 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.011083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:34.566076 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.566056    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:34.566200 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:34.566168    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:34.669329 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.669303    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 15 12:20:34.690737 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.690707    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"4ab38a209e39a73d849f483feaf369c25c70d377f390275f23f603bd5c8e07fe"}
Jul 15 12:20:34.692802 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.692730    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9brfz" event={"ID":"678138d5-cba2-4e9f-b8b0-a492d7eca657","Type":"ContainerStarted","Data":"d7225a25aae00e4dba54b7a78e654f117f87fe3bb560a13745ec4747b1c21a11"}
Jul 15 12:20:34.694931 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.694877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89" event={"ID":"c6a26546-f70d-4359-b5e9-2cf155a7a2b9","Type":"ContainerStarted","Data":"53ec38ecf332de5d8a515e689ae439754beb3f30b6071d72412ba83d87b9d0cb"}
Jul 15 12:20:34.695764 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.695743    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:34.696081 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.696060    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wswpt"
Jul 15 12:20:34.711742 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:34.711702    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9brfz" podStartSLOduration=5.170973963 podStartE2EDuration="23.711687901s" totalImagesPullingTime="18.540713938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:34.711173129 +0000 UTC m=+23.587896136" watchObservedRunningTime="2026-07-15 12:20:34.711687901 +0000 UTC m=+23.588410908"
Jul 15 12:20:35.500942 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.500591    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T12:20:34.669325007Z","UUID":"36aa975d-7571-45c7-9794-e9e5f380780d","Handler":null,"Name":"","Endpoint":""}
Jul 15 12:20:35.502525 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.502501    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 15 12:20:35.502676 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.502564    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 15 12:20:35.566802 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.566768    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:35.566995 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.566820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:35.566995 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:35.566920    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:35.567108 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:35.567048    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:35.700136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700107    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:20:35.700848 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700440    2576 generic.go:356] "Generic (PLEG): container finished" podID="c87a0dff-b6af-428d-b7fc-0091fa66d911" containerID="aa6ed2428c13ecaf78c917ce217d7aa4567490a83776d07f9f693433cb643abf" exitCode=1
Jul 15 12:20:35.700848 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"2540d9d4bba6f54d7f279e0d2e77f83c2bd90655781e5a2b2318bf1fe7093162"}
Jul 15 12:20:35.700848 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"8520bde1b9c080b40f31b482fd856347ba3dd21954d363241ac584426348716e"}
Jul 15 12:20:35.700848 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700685    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"66558cefcfbc59bfb1f0063d9498c63b857cd21a519aec22514b8b3e2a3b8a3f"}
Jul 15 12:20:35.700848 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700742    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"6ef18be8b1451a551c194913aa1bb37fbb2c98f021b2e82bfd18c955261977e3"}
Jul 15 12:20:35.700848 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:35.700788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerDied","Data":"aa6ed2428c13ecaf78c917ce217d7aa4567490a83776d07f9f693433cb643abf"}
Jul 15 12:20:36.566472 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:36.566445    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:36.566718 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:36.566551    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:36.705053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:36.705008    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89" event={"ID":"c6a26546-f70d-4359-b5e9-2cf155a7a2b9","Type":"ContainerStarted","Data":"9e0c26ce2e0a4ac67abb18111065ca15af42fe1385e796e396eb6916c547a102"}
Jul 15 12:20:36.726066 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:36.726016    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jzb89" podStartSLOduration=4.698308915 podStartE2EDuration="25.725996829s" totalImagesPullingTime="21.027687914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:36.725848236 +0000 UTC m=+25.602571236" watchObservedRunningTime="2026-07-15 12:20:36.725996829 +0000 UTC m=+25.602719935"
Jul 15 12:20:37.566430 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:37.566347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:37.566610 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:37.566347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:37.566610 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:37.566500    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:37.566735 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:37.566603    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:38.566755 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:38.566718    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:38.567347 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:38.566818    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:38.712878 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:38.712849    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:20:38.713279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:38.713255    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"e14909e18ff2a2f21af9b88df72ef7ab3f7edd69c21d413161431100831e28d1"}
Jul 15 12:20:38.714805 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:38.714780    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e01600-0295-4781-b1e1-5825eb79bad8" containerID="2faf7de25f5425d85e2b2451aabda27663cfa669052406707428d61406addd3b" exitCode=0
Jul 15 12:20:38.714901 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:38.714823    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerDied","Data":"2faf7de25f5425d85e2b2451aabda27663cfa669052406707428d61406addd3b"}
Jul 15 12:20:39.566986 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:39.566954    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:39.567354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:39.566960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:39.567354 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:39.567073    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:39.567354 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:39.567124    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:40.566713 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.566450    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:40.566868 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:40.566715    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:40.720702 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.720666    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e01600-0295-4781-b1e1-5825eb79bad8" containerID="96954f34e60e60972952453d1c9c8740a720abc623e1278fd36b529c7d4b6090" exitCode=0
Jul 15 12:20:40.721159 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.720740    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerDied","Data":"96954f34e60e60972952453d1c9c8740a720abc623e1278fd36b529c7d4b6090"}
Jul 15 12:20:40.724232 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.724214    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:20:40.724562 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.724526    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"d6bac390dd16f418c5f659bd1c50cc68615a818f949266d0d88fc10b1c9a588f"}
Jul 15 12:20:40.724754 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.724737    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:40.724834 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.724776    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:40.724935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.724920    2576 scope.go:117] "RemoveContainer" containerID="aa6ed2428c13ecaf78c917ce217d7aa4567490a83776d07f9f693433cb643abf"
Jul 15 12:20:40.740913 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:40.740895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:41.568249 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.568199    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:41.568414 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.568286    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:41.568414 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:41.568323    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:41.568414 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:41.568393    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:41.730315 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.730288    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:20:41.730744 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.730671    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" event={"ID":"c87a0dff-b6af-428d-b7fc-0091fa66d911","Type":"ContainerStarted","Data":"571e08b064aef75c4935118c0ad34591b0c2698b51eabba57c458eeaf38650e4"}
Jul 15 12:20:41.730847 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.730829    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:41.745924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.745895    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:20:41.762354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:41.762310    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw" podStartSLOduration=10.822308494 podStartE2EDuration="30.762293928s" totalImagesPullingTime="19.939985434s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:41.761768579 +0000 UTC m=+30.638491598" watchObservedRunningTime="2026-07-15 12:20:41.762293928 +0000 UTC m=+30.639016975"
Jul 15 12:20:42.566904 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.566652    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:42.567010 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:42.566981    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:42.735809 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.735762    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e01600-0295-4781-b1e1-5825eb79bad8" containerID="f027dfe24cd6c7932aa9a1e909e6f93225d5a69778bf49dfcc014791a2a4e2b7" exitCode=0
Jul 15 12:20:42.736109 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.735827    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerDied","Data":"f027dfe24cd6c7932aa9a1e909e6f93225d5a69778bf49dfcc014791a2a4e2b7"}
Jul 15 12:20:42.781079 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.781054    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mbqqd"]
Jul 15 12:20:42.781193 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.781147    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:42.781263 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:42.781223    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:42.784101 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.784079    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjzm8"]
Jul 15 12:20:42.784207 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.784171    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:42.784275 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:42.784256    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:42.788281 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.788263    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7kfpt"]
Jul 15 12:20:42.788352 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:42.788337    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:42.788408 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:42.788396    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:44.566553 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:44.566500    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:44.567487 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:44.566645    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:44.567487 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:44.566657    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:44.567487 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:44.566741    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:44.567487 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:44.566804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:44.567487 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:44.566906    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:45.289592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:45.289559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:45.289834 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.289661    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:45.289834 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:45.289672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:45.289834 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.289729    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret podName:ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:17.289709229 +0000 UTC m=+66.166432221 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret") pod "global-pull-secret-syncer-7kfpt" (UID: "ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6") : object "kube-system"/"original-pull-secret" not registered
Jul 15 12:20:45.289834 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.289778    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:45.289834 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.289835    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs podName:877ad258-985f-4f41-ba9b-df2d9e3b4133 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:17.289817523 +0000 UTC m=+66.166540513 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs") pod "network-metrics-daemon-zjzm8" (UID: "877ad258-985f-4f41-ba9b-df2d9e3b4133") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 15 12:20:45.390967 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:45.390926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:45.391159 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.391113    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 15 12:20:45.391272 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.391179    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 15 12:20:45.391272 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.391196    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xtt4k for pod openshift-network-diagnostics/network-check-target-mbqqd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:45.391272 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:45.391247    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k podName:6f21eba7-e283-4410-9cb4-db8d67d85479 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:17.391233966 +0000 UTC m=+66.267956957 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xtt4k" (UniqueName: "kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k") pod "network-check-target-mbqqd" (UID: "6f21eba7-e283-4410-9cb4-db8d67d85479") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 15 12:20:46.566341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:46.566301    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:46.566341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:46.566331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:46.566798 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:46.566315    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:46.566798 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:46.566423    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7kfpt" podUID="ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6"
Jul 15 12:20:46.566798 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:46.566506    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mbqqd" podUID="6f21eba7-e283-4410-9cb4-db8d67d85479"
Jul 15 12:20:46.566798 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:46.566609    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zjzm8" podUID="877ad258-985f-4f41-ba9b-df2d9e3b4133"
Jul 15 12:20:46.965109 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:46.965086    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-6.ec2.internal" event="NodeReady"
Jul 15 12:20:46.965259 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:46.965211    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 15 12:20:47.020565 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.020522    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6dfb95dbb5-qswrz"]
Jul 15 12:20:47.024309 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.024291    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"]
Jul 15 12:20:47.024429 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.024414    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.026983 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.026967    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-m78dh"]
Jul 15 12:20:47.027087 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.027072    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"
Jul 15 12:20:47.027732 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.027718    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 15 12:20:47.027785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.027720    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 15 12:20:47.029695 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.029675    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.030279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.030264    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"]
Jul 15 12:20:47.032776 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.032757    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.033082 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.033043    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 15 12:20:47.036919 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.036899    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"]
Jul 15 12:20:47.039213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.039195    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 15 12:20:47.039999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.039688    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 15 12:20:47.039999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.039697    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.040147 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.040035    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-8ncs8\""
Jul 15 12:20:47.040257 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.040242    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-qx2sl\""
Jul 15 12:20:47.040321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.040306    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 15 12:20:47.040414 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.040401    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 15 12:20:47.040562 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.040547    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 15 12:20:47.041169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.041153    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 15 12:20:47.041725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.041524    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:20:47.041725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.041575    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zptxr\""
Jul 15 12:20:47.041725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.041619    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 15 12:20:47.041725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.041630    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4bnx5\""
Jul 15 12:20:47.048473 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.048424    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:20:47.048649 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.048632    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 15 12:20:47.048736 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.048635    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 15 12:20:47.048736 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.048656    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 15 12:20:47.049249 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.049227    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jlqcz\""
Jul 15 12:20:47.049564 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.049551    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"]
Jul 15 12:20:47.051404 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.050354    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 15 12:20:47.052554 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.051824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-m78dh"]
Jul 15 12:20:47.078636 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.078515    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"]
Jul 15 12:20:47.082937 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.082916    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dfb95dbb5-qswrz"]
Jul 15 12:20:47.092793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.092775    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"]
Jul 15 12:20:47.093776 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.093761    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pwmnc"]
Jul 15 12:20:47.096870 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.096850    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.101604 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.101574    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 15 12:20:47.101701 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.101649    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zgxhc"]
Jul 15 12:20:47.101701 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.101682    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 15 12:20:47.103200 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.103181    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 15 12:20:47.104433 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.104416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.105911 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.105889    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bx6xw\""
Jul 15 12:20:47.106554 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.106520    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hw8dx\""
Jul 15 12:20:47.106643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.106564    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 15 12:20:47.107075 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.107056    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 15 12:20:47.114507 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.114488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pwmnc"]
Jul 15 12:20:47.133872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.133841    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zgxhc"]
Jul 15 12:20:47.201748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201728    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdec6837-d575-487c-8e03-4855bf81c163-ca-trust-extracted\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.201845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g95rk\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-kube-api-access-g95rk\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.201845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201773    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.201845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201791    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95aa8e11-cade-4b79-bd27-10d2459910a4-config-volume\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.201845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg2pb\" (UniqueName: \"kubernetes.io/projected/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-kube-api-access-rg2pb\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.201845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-trusted-ca\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.201997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201860    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.201997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95aa8e11-cade-4b79-bd27-10d2459910a4-tmp-dir\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.201997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201906    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rrvk\" (UniqueName: \"kubernetes.io/projected/702ba535-61f8-426e-8158-901f150179ce-kube-api-access-2rrvk\") pod \"network-check-source-869dcfcdcd-p9kpt\" (UID: \"702ba535-61f8-426e-8158-901f150179ce\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"
Jul 15 12:20:47.201997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-bound-sa-token\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.201997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201972    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnsqs\" (UniqueName: \"kubernetes.io/projected/95aa8e11-cade-4b79-bd27-10d2459910a4-kube-api-access-xnsqs\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.201997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.201996    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwwzg\" (UniqueName: \"kubernetes.io/projected/a560a9c7-677a-4ad2-9b65-1e15843b171d-kube-api-access-nwwzg\") pod \"ingress-canary-pwmnc\" (UID: \"a560a9c7-677a-4ad2-9b65-1e15843b171d\") " pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.202199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202021    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-installation-pull-secrets\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.202199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202042    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b3b94358-208c-4e52-9d82-b7de8a12e86d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.202199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.202199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.202199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-registry-certificates\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.202365 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202211    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41151eeb-4c7c-4839-95ed-2f54de1b0360-config\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.202365 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202234    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a560a9c7-677a-4ad2-9b65-1e15843b171d-cert\") pod \"ingress-canary-pwmnc\" (UID: \"a560a9c7-677a-4ad2-9b65-1e15843b171d\") " pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.202365 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.202365 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202282    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-image-registry-private-configuration\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.202365 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202305    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41151eeb-4c7c-4839-95ed-2f54de1b0360-serving-cert\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.202365 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.202321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr8ml\" (UniqueName: \"kubernetes.io/projected/41151eeb-4c7c-4839-95ed-2f54de1b0360-kube-api-access-zr8ml\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.302996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.302933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41151eeb-4c7c-4839-95ed-2f54de1b0360-serving-cert\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.302996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.302962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr8ml\" (UniqueName: \"kubernetes.io/projected/41151eeb-4c7c-4839-95ed-2f54de1b0360-kube-api-access-zr8ml\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.302996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.302987    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdec6837-d575-487c-8e03-4855bf81c163-ca-trust-extracted\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303057    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g95rk\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-kube-api-access-g95rk\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303092    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.303222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303124    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95aa8e11-cade-4b79-bd27-10d2459910a4-config-volume\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.303222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303153    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg2pb\" (UniqueName: \"kubernetes.io/projected/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-kube-api-access-rg2pb\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.303222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303182    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-trusted-ca\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdec6837-d575-487c-8e03-4855bf81c163-ca-trust-extracted\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.303445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303349    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95aa8e11-cade-4b79-bd27-10d2459910a4-tmp-dir\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.303445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303387    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rrvk\" (UniqueName: \"kubernetes.io/projected/702ba535-61f8-426e-8158-901f150179ce-kube-api-access-2rrvk\") pod \"network-check-source-869dcfcdcd-p9kpt\" (UID: \"702ba535-61f8-426e-8158-901f150179ce\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"
Jul 15 12:20:47.303445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-bound-sa-token\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnsqs\" (UniqueName: \"kubernetes.io/projected/95aa8e11-cade-4b79-bd27-10d2459910a4-kube-api-access-xnsqs\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.303476    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwwzg\" (UniqueName: \"kubernetes.io/projected/a560a9c7-677a-4ad2-9b65-1e15843b171d-kube-api-access-nwwzg\") pod \"ingress-canary-pwmnc\" (UID: \"a560a9c7-677a-4ad2-9b65-1e15843b171d\") " pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-installation-pull-secrets\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.303563    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls podName:95aa8e11-cade-4b79-bd27-10d2459910a4 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:47.803524661 +0000 UTC m=+36.680247645 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls") pod "dns-default-zgxhc" (UID: "95aa8e11-cade-4b79-bd27-10d2459910a4") : secret "dns-default-metrics-tls" not found
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303597    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b3b94358-208c-4e52-9d82-b7de8a12e86d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.303700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-registry-certificates\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303743    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41151eeb-4c7c-4839-95ed-2f54de1b0360-config\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a560a9c7-677a-4ad2-9b65-1e15843b171d-cert\") pod \"ingress-canary-pwmnc\" (UID: \"a560a9c7-677a-4ad2-9b65-1e15843b171d\") " pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-image-registry-private-configuration\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95aa8e11-cade-4b79-bd27-10d2459910a4-config-volume\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.304115 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.303963    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95aa8e11-cade-4b79-bd27-10d2459910a4-tmp-dir\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.304393 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.304125    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 15 12:20:47.304393 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.304172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert podName:b3b94358-208c-4e52-9d82-b7de8a12e86d nodeName:}" failed. No retries permitted until 2026-07-15 12:20:47.804157365 +0000 UTC m=+36.680880354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-m78dh" (UID: "b3b94358-208c-4e52-9d82-b7de8a12e86d") : secret "networking-console-plugin-cert" not found
Jul 15 12:20:47.304393 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.304355    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-registry-certificates\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.304393 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.304379    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-trusted-ca\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.304706 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.304682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/41151eeb-4c7c-4839-95ed-2f54de1b0360-config\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.304784 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.304772    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:20:47.304837 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.304786    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dfb95dbb5-qswrz: secret "image-registry-tls" not found
Jul 15 12:20:47.304837 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.304822    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls podName:bdec6837-d575-487c-8e03-4855bf81c163 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:47.804809479 +0000 UTC m=+36.681532463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls") pod "image-registry-6dfb95dbb5-qswrz" (UID: "bdec6837-d575-487c-8e03-4855bf81c163") : secret "image-registry-tls" not found
Jul 15 12:20:47.304993 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.304950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b3b94358-208c-4e52-9d82-b7de8a12e86d-nginx-conf\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.307557 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.307419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/41151eeb-4c7c-4839-95ed-2f54de1b0360-serving-cert\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.307644 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.307553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-image-registry-private-configuration\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.307644 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.307490    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-installation-pull-secrets\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.307644 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.307461    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.307746 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.307650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a560a9c7-677a-4ad2-9b65-1e15843b171d-cert\") pod \"ingress-canary-pwmnc\" (UID: \"a560a9c7-677a-4ad2-9b65-1e15843b171d\") " pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.316151 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.316127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnsqs\" (UniqueName: \"kubernetes.io/projected/95aa8e11-cade-4b79-bd27-10d2459910a4-kube-api-access-xnsqs\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.316475 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.316453    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg2pb\" (UniqueName: \"kubernetes.io/projected/c74f7ff8-ae45-4bf9-ad15-59abca5df8b2-kube-api-access-rg2pb\") pod \"kube-storage-version-migrator-operator-799cb7f958-fmzhh\" (UID: \"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.316691 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.316673    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rrvk\" (UniqueName: \"kubernetes.io/projected/702ba535-61f8-426e-8158-901f150179ce-kube-api-access-2rrvk\") pod \"network-check-source-869dcfcdcd-p9kpt\" (UID: \"702ba535-61f8-426e-8158-901f150179ce\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"
Jul 15 12:20:47.317791 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.317762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr8ml\" (UniqueName: \"kubernetes.io/projected/41151eeb-4c7c-4839-95ed-2f54de1b0360-kube-api-access-zr8ml\") pod \"service-ca-operator-7695fc6c8c-tsd4t\" (UID: \"41151eeb-4c7c-4839-95ed-2f54de1b0360\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.318020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.317994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g95rk\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-kube-api-access-g95rk\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.319188 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.319165    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-bound-sa-token\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.320092 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.320077    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwwzg\" (UniqueName: \"kubernetes.io/projected/a560a9c7-677a-4ad2-9b65-1e15843b171d-kube-api-access-nwwzg\") pod \"ingress-canary-pwmnc\" (UID: \"a560a9c7-677a-4ad2-9b65-1e15843b171d\") " pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.336199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.336163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wswpt_33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77/konnectivity-agent/0.log" limit=1024
Jul 15 12:20:47.363085 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.363065    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"
Jul 15 12:20:47.374386 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.374365    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"
Jul 15 12:20:47.380202 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.380181    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"
Jul 15 12:20:47.405089 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.405061    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pwmnc"
Jul 15 12:20:47.526212 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.526184    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt"]
Jul 15 12:20:47.530328 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.530262    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t"]
Jul 15 12:20:47.530644 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:47.530617    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702ba535_61f8_426e_8158_901f150179ce.slice/crio-df875d929e98d5a2e9f6a0dc1e090901473692d76b75f4ce774ec9afcfa03a14 WatchSource:0}: Error finding container df875d929e98d5a2e9f6a0dc1e090901473692d76b75f4ce774ec9afcfa03a14: Status 404 returned error can't find the container with id df875d929e98d5a2e9f6a0dc1e090901473692d76b75f4ce774ec9afcfa03a14
Jul 15 12:20:47.533726 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:47.533705    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41151eeb_4c7c_4839_95ed_2f54de1b0360.slice/crio-24e694f670136e9a5e69018b8a22d35de1e8c017d48bf56267f83579315e6180 WatchSource:0}: Error finding container 24e694f670136e9a5e69018b8a22d35de1e8c017d48bf56267f83579315e6180: Status 404 returned error can't find the container with id 24e694f670136e9a5e69018b8a22d35de1e8c017d48bf56267f83579315e6180
Jul 15 12:20:47.538445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.538422    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh"]
Jul 15 12:20:47.541576 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:47.541551    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc74f7ff8_ae45_4bf9_ad15_59abca5df8b2.slice/crio-bafb593f099610efa55081965150fb5ca56a7391d805c915d87c8f2b276ccc6d WatchSource:0}: Error finding container bafb593f099610efa55081965150fb5ca56a7391d805c915d87c8f2b276ccc6d: Status 404 returned error can't find the container with id bafb593f099610efa55081965150fb5ca56a7391d805c915d87c8f2b276ccc6d
Jul 15 12:20:47.556656 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.556585    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pwmnc"]
Jul 15 12:20:47.559236 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:47.559220    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda560a9c7_677a_4ad2_9b65_1e15843b171d.slice/crio-b14de77a55abd34c8ae490d5ca872e3e596e89a59f299ca4711dda868d0b20ca WatchSource:0}: Error finding container b14de77a55abd34c8ae490d5ca872e3e596e89a59f299ca4711dda868d0b20ca: Status 404 returned error can't find the container with id b14de77a55abd34c8ae490d5ca872e3e596e89a59f299ca4711dda868d0b20ca
Jul 15 12:20:47.749955 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.749920    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pwmnc" event={"ID":"a560a9c7-677a-4ad2-9b65-1e15843b171d","Type":"ContainerStarted","Data":"b14de77a55abd34c8ae490d5ca872e3e596e89a59f299ca4711dda868d0b20ca"}
Jul 15 12:20:47.751007 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.750982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t" event={"ID":"41151eeb-4c7c-4839-95ed-2f54de1b0360","Type":"ContainerStarted","Data":"24e694f670136e9a5e69018b8a22d35de1e8c017d48bf56267f83579315e6180"}
Jul 15 12:20:47.753340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.753319    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e01600-0295-4781-b1e1-5825eb79bad8" containerID="c8ce2dc547218c92b095e82644bbbdcc084d2db671fc9538d9da1d113a9c5829" exitCode=0
Jul 15 12:20:47.753439 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.753385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerDied","Data":"c8ce2dc547218c92b095e82644bbbdcc084d2db671fc9538d9da1d113a9c5829"}
Jul 15 12:20:47.754560 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.754516    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt" event={"ID":"702ba535-61f8-426e-8158-901f150179ce","Type":"ContainerStarted","Data":"df875d929e98d5a2e9f6a0dc1e090901473692d76b75f4ce774ec9afcfa03a14"}
Jul 15 12:20:47.760942 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.760907    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh" event={"ID":"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2","Type":"ContainerStarted","Data":"bafb593f099610efa55081965150fb5ca56a7391d805c915d87c8f2b276ccc6d"}
Jul 15 12:20:47.801521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.801489    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wswpt_33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77/konnectivity-agent/0.log" limit=1024
Jul 15 12:20:47.806996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.806949    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:47.806996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.806993    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:47.807106 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:47.807026    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:47.807106 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807086    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:20:47.807106 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807102    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dfb95dbb5-qswrz: secret "image-registry-tls" not found
Jul 15 12:20:47.807106 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807102    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:20:47.807263 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807147    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls podName:bdec6837-d575-487c-8e03-4855bf81c163 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:48.80713362 +0000 UTC m=+37.683856608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls") pod "image-registry-6dfb95dbb5-qswrz" (UID: "bdec6837-d575-487c-8e03-4855bf81c163") : secret "image-registry-tls" not found
Jul 15 12:20:47.807263 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807147    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 15 12:20:47.807263 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807159    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls podName:95aa8e11-cade-4b79-bd27-10d2459910a4 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:48.807153739 +0000 UTC m=+37.683876723 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls") pod "dns-default-zgxhc" (UID: "95aa8e11-cade-4b79-bd27-10d2459910a4") : secret "dns-default-metrics-tls" not found
Jul 15 12:20:47.807263 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:47.807199    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert podName:b3b94358-208c-4e52-9d82-b7de8a12e86d nodeName:}" failed. No retries permitted until 2026-07-15 12:20:48.807182457 +0000 UTC m=+37.683905460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-m78dh" (UID: "b3b94358-208c-4e52-9d82-b7de8a12e86d") : secret "networking-console-plugin-cert" not found
Jul 15 12:20:48.567071 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.567039    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:20:48.567071 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.567079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:20:48.567294 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.567039    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:20:48.570600 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.570578    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 12:20:48.570974 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.570953    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nnkkc\""
Jul 15 12:20:48.571517 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.571497    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 12:20:48.571786 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.571762    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pcnmv\""
Jul 15 12:20:48.770785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.770749    2576 generic.go:356] "Generic (PLEG): container finished" podID="c3e01600-0295-4781-b1e1-5825eb79bad8" containerID="d634cf23b5d8328db945fe506834c2c2ee59ddc4d95dab624b0e72b4fa56fd4d" exitCode=0
Jul 15 12:20:48.771226 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.770826    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerDied","Data":"d634cf23b5d8328db945fe506834c2c2ee59ddc4d95dab624b0e72b4fa56fd4d"}
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.816467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.816772    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:48.816834    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.816951    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.817008    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert podName:b3b94358-208c-4e52-9d82-b7de8a12e86d nodeName:}" failed. No retries permitted until 2026-07-15 12:20:50.816990193 +0000 UTC m=+39.693713181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-m78dh" (UID: "b3b94358-208c-4e52-9d82-b7de8a12e86d") : secret "networking-console-plugin-cert" not found
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.817359    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.817404    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls podName:95aa8e11-cade-4b79-bd27-10d2459910a4 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:50.81738909 +0000 UTC m=+39.694112074 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls") pod "dns-default-zgxhc" (UID: "95aa8e11-cade-4b79-bd27-10d2459910a4") : secret "dns-default-metrics-tls" not found
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.817465    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.817476    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dfb95dbb5-qswrz: secret "image-registry-tls" not found
Jul 15 12:20:48.817591 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:48.817508    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls podName:bdec6837-d575-487c-8e03-4855bf81c163 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:50.817497685 +0000 UTC m=+39.694220670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls") pod "image-registry-6dfb95dbb5-qswrz" (UID: "bdec6837-d575-487c-8e03-4855bf81c163") : secret "image-registry-tls" not found
Jul 15 12:20:49.025210 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:49.025166    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wswpt_33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77/konnectivity-agent/0.log" limit=1024
Jul 15 12:20:49.776816 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:49.776787    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" event={"ID":"c3e01600-0295-4781-b1e1-5825eb79bad8","Type":"ContainerStarted","Data":"77358b7284a1f6d9a44395c2d065e97f6a48a49e67cb3de855790c7b5499e435"}
Jul 15 12:20:49.813520 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:49.813476    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kkfz5" podStartSLOduration=8.962947233 podStartE2EDuration="38.813460716s" totalImagesPullingTime="29.850513483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:49.811462061 +0000 UTC m=+38.688185068" watchObservedRunningTime="2026-07-15 12:20:49.813460716 +0000 UTC m=+38.690183721"
Jul 15 12:20:50.834084 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:50.834050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:50.834099    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:50.834141    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834202    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834224    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dfb95dbb5-qswrz: secret "image-registry-tls" not found
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834237    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834239    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834280    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls podName:95aa8e11-cade-4b79-bd27-10d2459910a4 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:54.83426805 +0000 UTC m=+43.710991034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls") pod "dns-default-zgxhc" (UID: "95aa8e11-cade-4b79-bd27-10d2459910a4") : secret "dns-default-metrics-tls" not found
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834304    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert podName:b3b94358-208c-4e52-9d82-b7de8a12e86d nodeName:}" failed. No retries permitted until 2026-07-15 12:20:54.834286842 +0000 UTC m=+43.711009826 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-m78dh" (UID: "b3b94358-208c-4e52-9d82-b7de8a12e86d") : secret "networking-console-plugin-cert" not found
Jul 15 12:20:50.834547 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:50.834327    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls podName:bdec6837-d575-487c-8e03-4855bf81c163 nodeName:}" failed. No retries permitted until 2026-07-15 12:20:54.834312074 +0000 UTC m=+43.711035057 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls") pod "image-registry-6dfb95dbb5-qswrz" (UID: "bdec6837-d575-487c-8e03-4855bf81c163") : secret "image-registry-tls" not found
Jul 15 12:20:53.788691 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.788654    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pwmnc" event={"ID":"a560a9c7-677a-4ad2-9b65-1e15843b171d","Type":"ContainerStarted","Data":"f6ebb8cc1c9f6b15cba47a47d937b0fed87e712800e7aea79354571521120bd3"}
Jul 15 12:20:53.790580 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.790521    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t" event={"ID":"41151eeb-4c7c-4839-95ed-2f54de1b0360","Type":"ContainerStarted","Data":"335e5e330245e2cfff665c8509d0c2b97ca0045b419c66b1350d2c9d0b36de30"}
Jul 15 12:20:53.792178 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.792146    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt" event={"ID":"702ba535-61f8-426e-8158-901f150179ce","Type":"ContainerStarted","Data":"f0831d2f5c7fa7d3af99fc88fb29557c7db722b67dab9dda9a97007dea3fa0cb"}
Jul 15 12:20:53.793632 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.793611    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh" event={"ID":"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2","Type":"ContainerStarted","Data":"80fb8f117b2c5b707493f6560bcfb083ef68b1ab5889d680400c39a7481a391c"}
Jul 15 12:20:53.814255 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.814217    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pwmnc" podStartSLOduration=1.292955478 podStartE2EDuration="6.814204052s" totalImagesPullingTime="5.521248574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:53.812795551 +0000 UTC m=+42.689518557" watchObservedRunningTime="2026-07-15 12:20:53.814204052 +0000 UTC m=+42.690927036"
Jul 15 12:20:53.831986 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.831937    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t" podStartSLOduration=34.27829516 podStartE2EDuration="39.831921659s" totalImagesPullingTime="5.553626499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:53.830715638 +0000 UTC m=+42.707438646" watchObservedRunningTime="2026-07-15 12:20:53.831921659 +0000 UTC m=+42.708644666"
Jul 15 12:20:53.858214 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.858167    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-p9kpt" podStartSLOduration=33.307534038 podStartE2EDuration="38.858154065s" totalImagesPullingTime="5.550620027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:53.857626613 +0000 UTC m=+42.734349620" watchObservedRunningTime="2026-07-15 12:20:53.858154065 +0000 UTC m=+42.734877070"
Jul 15 12:20:53.885047 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:53.884994    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh" podStartSLOduration=36.34578628 podStartE2EDuration="41.884979388s" totalImagesPullingTime="5.539193108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:53.884805939 +0000 UTC m=+42.761528937" watchObservedRunningTime="2026-07-15 12:20:53.884979388 +0000 UTC m=+42.761702397"
Jul 15 12:20:54.250612 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.250581    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"]
Jul 15 12:20:54.277484 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.277453    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"]
Jul 15 12:20:54.277659 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.277593    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"
Jul 15 12:20:54.280309 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.280286    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 15 12:20:54.280547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.280516    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2fpwp\""
Jul 15 12:20:54.280725 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.280699    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 15 12:20:54.464549 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.464461    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwgr8\" (UniqueName: \"kubernetes.io/projected/b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b-kube-api-access-xwgr8\") pod \"migrator-7f59b54f6c-j4mvm\" (UID: \"b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"
Jul 15 12:20:54.565143 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.565060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwgr8\" (UniqueName: \"kubernetes.io/projected/b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b-kube-api-access-xwgr8\") pod \"migrator-7f59b54f6c-j4mvm\" (UID: \"b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"
Jul 15 12:20:54.576001 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.575971    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwgr8\" (UniqueName: \"kubernetes.io/projected/b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b-kube-api-access-xwgr8\") pod \"migrator-7f59b54f6c-j4mvm\" (UID: \"b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"
Jul 15 12:20:54.587936 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.587912    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"
Jul 15 12:20:54.741847 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.741823    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm"]
Jul 15 12:20:54.745279 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:54.745246    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34d1a1c_3647_4c4e_a47f_f23bb2b1e18b.slice/crio-0d31d96c2102522e006a9a620b0518ca11485dd96d09724b73400ae0b19bebce WatchSource:0}: Error finding container 0d31d96c2102522e006a9a620b0518ca11485dd96d09724b73400ae0b19bebce: Status 404 returned error can't find the container with id 0d31d96c2102522e006a9a620b0518ca11485dd96d09724b73400ae0b19bebce
Jul 15 12:20:54.797081 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.797049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm" event={"ID":"b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b","Type":"ContainerStarted","Data":"0d31d96c2102522e006a9a620b0518ca11485dd96d09724b73400ae0b19bebce"}
Jul 15 12:20:54.867968 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.867904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:20:54.867968 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.867961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:54.867996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868042    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868076    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868095    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls podName:95aa8e11-cade-4b79-bd27-10d2459910a4 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:02.868080966 +0000 UTC m=+51.744803950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls") pod "dns-default-zgxhc" (UID: "95aa8e11-cade-4b79-bd27-10d2459910a4") : secret "dns-default-metrics-tls" not found
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868117    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert podName:b3b94358-208c-4e52-9d82-b7de8a12e86d nodeName:}" failed. No retries permitted until 2026-07-15 12:21:02.868106458 +0000 UTC m=+51.744829442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-m78dh" (UID: "b3b94358-208c-4e52-9d82-b7de8a12e86d") : secret "networking-console-plugin-cert" not found
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868120    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:20:54.868140 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868137    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dfb95dbb5-qswrz: secret "image-registry-tls" not found
Jul 15 12:20:54.868384 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:20:54.868188    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls podName:bdec6837-d575-487c-8e03-4855bf81c163 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:02.868170661 +0000 UTC m=+51.744893655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls") pod "image-registry-6dfb95dbb5-qswrz" (UID: "bdec6837-d575-487c-8e03-4855bf81c163") : secret "image-registry-tls" not found
Jul 15 12:20:56.804455 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:56.804385    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm" event={"ID":"b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b","Type":"ContainerStarted","Data":"e90a093aef31a307bcbe44b43d36c4f07e6a4cd50bbd8d19fb1f435fabf1a741"}
Jul 15 12:20:56.804455 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:56.804450    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm" event={"ID":"b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b","Type":"ContainerStarted","Data":"125123a0c6ba5d1865d1c3b57103ac72ffc66426f83c959d3ffaa3822c60e307"}
Jul 15 12:20:56.821173 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:56.821128    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-j4mvm" podStartSLOduration=1.0841967829999999 podStartE2EDuration="2.821114939s" totalImagesPullingTime="1.736918156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:56.819949348 +0000 UTC m=+45.696672355" watchObservedRunningTime="2026-07-15 12:20:56.821114939 +0000 UTC m=+45.697837944"
Jul 15 12:20:57.366471 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.366439    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-h54gb"]
Jul 15 12:20:57.406361 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.406333    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-h54gb"]
Jul 15 12:20:57.406503 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.406431    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.409181 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.409159    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 15 12:20:57.409320 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.409160    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 15 12:20:57.410189 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.410162    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-p4zpm\""
Jul 15 12:20:57.410298 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.410167    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 15 12:20:57.410298 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.410166    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 15 12:20:57.591179 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.591147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbsq9\" (UniqueName: \"kubernetes.io/projected/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-kube-api-access-kbsq9\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.591179 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.591179    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-signing-cabundle\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.591355 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.591226    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-signing-key\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.691709 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.691679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-signing-cabundle\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.691836 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.691824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-signing-key\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.691897 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.691886    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbsq9\" (UniqueName: \"kubernetes.io/projected/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-kube-api-access-kbsq9\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.692373 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.692350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-signing-cabundle\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.694143 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.694126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-signing-key\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.702771 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.702748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbsq9\" (UniqueName: \"kubernetes.io/projected/f93d5dd0-022c-42b2-bbf5-b5f6930aaa58-kube-api-access-kbsq9\") pod \"service-ca-79f68b9f4f-h54gb\" (UID: \"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58\") " pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.715401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.715382    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb"
Jul 15 12:20:57.855931 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:57.855899    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-h54gb"]
Jul 15 12:20:57.860708 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:20:57.860671    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf93d5dd0_022c_42b2_bbf5_b5f6930aaa58.slice/crio-273e90e8dfc1ba2b73d6adbf33ca85363e4deed88206b4d16ffa123e225fdfb6 WatchSource:0}: Error finding container 273e90e8dfc1ba2b73d6adbf33ca85363e4deed88206b4d16ffa123e225fdfb6: Status 404 returned error can't find the container with id 273e90e8dfc1ba2b73d6adbf33ca85363e4deed88206b4d16ffa123e225fdfb6
Jul 15 12:20:58.812357 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:58.812318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb" event={"ID":"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58","Type":"ContainerStarted","Data":"b81ea21fbd019e3c71f8b0276acce2b25faf64967300b869b44a8579bd10ca97"}
Jul 15 12:20:58.812357 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:58.812363    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb" event={"ID":"f93d5dd0-022c-42b2-bbf5-b5f6930aaa58","Type":"ContainerStarted","Data":"273e90e8dfc1ba2b73d6adbf33ca85363e4deed88206b4d16ffa123e225fdfb6"}
Jul 15 12:20:58.834818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:20:58.834780    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-h54gb" podStartSLOduration=1.834767325 podStartE2EDuration="1.834767325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:20:58.833352289 +0000 UTC m=+47.710075295" watchObservedRunningTime="2026-07-15 12:20:58.834767325 +0000 UTC m=+47.711490333"
Jul 15 12:21:02.928179 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:02.928147    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:02.928193    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:02.928218    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:02.928285    2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:02.928303    2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6dfb95dbb5-qswrz: secret "image-registry-tls" not found
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:02.928336    2576 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:02.928372    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls podName:bdec6837-d575-487c-8e03-4855bf81c163 nodeName:}" failed. No retries permitted until 2026-07-15 12:21:18.928352951 +0000 UTC m=+67.805075955 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls") pod "image-registry-6dfb95dbb5-qswrz" (UID: "bdec6837-d575-487c-8e03-4855bf81c163") : secret "image-registry-tls" not found
Jul 15 12:21:02.928676 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:02.928389    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert podName:b3b94358-208c-4e52-9d82-b7de8a12e86d nodeName:}" failed. No retries permitted until 2026-07-15 12:21:18.928380932 +0000 UTC m=+67.805103917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-m78dh" (UID: "b3b94358-208c-4e52-9d82-b7de8a12e86d") : secret "networking-console-plugin-cert" not found
Jul 15 12:21:02.930402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:02.930382    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95aa8e11-cade-4b79-bd27-10d2459910a4-metrics-tls\") pod \"dns-default-zgxhc\" (UID: \"95aa8e11-cade-4b79-bd27-10d2459910a4\") " pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:21:03.011837 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:03.011810    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:21:03.173443 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:03.173387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zgxhc"]
Jul 15 12:21:03.829512 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:03.829475    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zgxhc" event={"ID":"95aa8e11-cade-4b79-bd27-10d2459910a4","Type":"ContainerStarted","Data":"c14ba82e4f48912fc5bf0f2aaabbda67abbf9cb874fc7a0ddec71c2033e6fcad"}
Jul 15 12:21:06.841820 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:06.841785    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zgxhc" event={"ID":"95aa8e11-cade-4b79-bd27-10d2459910a4","Type":"ContainerStarted","Data":"c850f85730bf0c3e728381043ae36341f51863fc5dcf8a8dc34a221935878cdb"}
Jul 15 12:21:06.842297 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:06.841871    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zgxhc" event={"ID":"95aa8e11-cade-4b79-bd27-10d2459910a4","Type":"ContainerStarted","Data":"efa7b0c445168e8dc53b4bda38863710d95c93a597fe5dce0b3a897e8b055c69"}
Jul 15 12:21:06.842297 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:06.841924    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:21:13.748525 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:13.748498    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8fhzw"
Jul 15 12:21:13.803387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:13.803329    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zgxhc" podStartSLOduration=24.174090532 podStartE2EDuration="26.803312976s" totalImagesPullingTime="2.629222444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:06.864759435 +0000 UTC m=+55.741482505" watchObservedRunningTime="2026-07-15 12:21:13.803312976 +0000 UTC m=+62.680035981"
Jul 15 12:21:16.847225 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:16.847191    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zgxhc"
Jul 15 12:21:17.333275 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.333242    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:21:17.333437 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.333305    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:21:17.336444 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.336426    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 15 12:21:17.336491 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.336455    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 15 12:21:17.345733 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.345695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6-original-pull-secret\") pod \"global-pull-secret-syncer-7kfpt\" (UID: \"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6\") " pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:21:17.345834 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.345778    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/877ad258-985f-4f41-ba9b-df2d9e3b4133-metrics-certs\") pod \"network-metrics-daemon-zjzm8\" (UID: \"877ad258-985f-4f41-ba9b-df2d9e3b4133\") " pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:21:17.392201 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.392181    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pcnmv\""
Jul 15 12:21:17.398326 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.398312    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7kfpt"
Jul 15 12:21:17.399391 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.399378    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zjzm8"
Jul 15 12:21:17.434199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.434156    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:21:17.436460 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.436437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtt4k\" (UniqueName: \"kubernetes.io/projected/6f21eba7-e283-4410-9cb4-db8d67d85479-kube-api-access-xtt4k\") pod \"network-check-target-mbqqd\" (UID: \"6f21eba7-e283-4410-9cb4-db8d67d85479\") " pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:21:17.529829 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.529807    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zjzm8"]
Jul 15 12:21:17.532127 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:17.532098    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877ad258_985f_4f41_ba9b_df2d9e3b4133.slice/crio-205b188a87a3137e653fc7e292db8ef252cca96283c617a96274692e9748c2d4 WatchSource:0}: Error finding container 205b188a87a3137e653fc7e292db8ef252cca96283c617a96274692e9748c2d4: Status 404 returned error can't find the container with id 205b188a87a3137e653fc7e292db8ef252cca96283c617a96274692e9748c2d4
Jul 15 12:21:17.540797 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.540776    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7kfpt"]
Jul 15 12:21:17.543447 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:17.543427    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac68a54c_2842_44d0_b0a5_5fa81fd0e1e6.slice/crio-13f1136d20cdb4d4962543003fcd67a158c4703af03d828c7bd9785e1278b693 WatchSource:0}: Error finding container 13f1136d20cdb4d4962543003fcd67a158c4703af03d828c7bd9785e1278b693: Status 404 returned error can't find the container with id 13f1136d20cdb4d4962543003fcd67a158c4703af03d828c7bd9785e1278b693
Jul 15 12:21:17.683767 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.683745    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nnkkc\""
Jul 15 12:21:17.691874 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.691851    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:21:17.818108 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.818071    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mbqqd"]
Jul 15 12:21:17.821361 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:17.821341    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f21eba7_e283_4410_9cb4_db8d67d85479.slice/crio-21e3c040b66e80c8dadf60b4ffd2131ebc692239af70cdb2eb2d14481fc8e1e8 WatchSource:0}: Error finding container 21e3c040b66e80c8dadf60b4ffd2131ebc692239af70cdb2eb2d14481fc8e1e8: Status 404 returned error can't find the container with id 21e3c040b66e80c8dadf60b4ffd2131ebc692239af70cdb2eb2d14481fc8e1e8
Jul 15 12:21:17.873344 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.873318    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mbqqd" event={"ID":"6f21eba7-e283-4410-9cb4-db8d67d85479","Type":"ContainerStarted","Data":"4f6e3345a2a425c2fe88170783722f2586feab570f487bc40db8eb9185eb4ced"}
Jul 15 12:21:17.873682 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.873370    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mbqqd" event={"ID":"6f21eba7-e283-4410-9cb4-db8d67d85479","Type":"ContainerStarted","Data":"21e3c040b66e80c8dadf60b4ffd2131ebc692239af70cdb2eb2d14481fc8e1e8"}
Jul 15 12:21:17.874432 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.874410    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjzm8" event={"ID":"877ad258-985f-4f41-ba9b-df2d9e3b4133","Type":"ContainerStarted","Data":"205b188a87a3137e653fc7e292db8ef252cca96283c617a96274692e9748c2d4"}
Jul 15 12:21:17.875433 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:17.875415    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7kfpt" event={"ID":"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6","Type":"ContainerStarted","Data":"13f1136d20cdb4d4962543003fcd67a158c4703af03d828c7bd9785e1278b693"}
Jul 15 12:21:18.089304 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.089231    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"]
Jul 15 12:21:18.092474 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.092448    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.098145 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.098123    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 15 12:21:18.098957 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.098472    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.098957 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.098580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 15 12:21:18.098957 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.098756    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.129130 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.129099    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zcxwn"]
Jul 15 12:21:18.132161 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.132138    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.132918 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.132795    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"]
Jul 15 12:21:18.137768 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.137568    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 15 12:21:18.138352 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.138255    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.138517 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.138499    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 15 12:21:18.139251 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.139230    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.140362 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.140161    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d328f198-1825-4446-825a-1bc047a4971c-tmp\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.140362 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.140255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d328f198-1825-4446-825a-1bc047a4971c-klusterlet-config\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.140362 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.140312    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q54c\" (UniqueName: \"kubernetes.io/projected/d328f198-1825-4446-825a-1bc047a4971c-kube-api-access-6q54c\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.140583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.140444    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-gxhg2\""
Jul 15 12:21:18.142939 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.142879    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-595544f54d-bxnpp"]
Jul 15 12:21:18.145850 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.145832    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.151699 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.151601    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.151969 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.151949    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qcszh\""
Jul 15 12:21:18.152080 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.152056    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.153038 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.152746    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 15 12:21:18.153432 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.153282    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 15 12:21:18.154334 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.154313    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 15 12:21:18.157213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.157164    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 15 12:21:18.158962 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.158943    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 15 12:21:18.163570 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.163552    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"]
Jul 15 12:21:18.167665 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.167646    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.175686 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.175424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zcxwn"]
Jul 15 12:21:18.176454 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.176247    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.176454 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.176363    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-x74md\""
Jul 15 12:21:18.176907 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.176572    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 15 12:21:18.176981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.176966    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.192183 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.192164    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"]
Jul 15 12:21:18.201045 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.201024    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"]
Jul 15 12:21:18.206681 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.206656    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-595544f54d-bxnpp"]
Jul 15 12:21:18.206914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.206815    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"
Jul 15 12:21:18.209510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.209490    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.209628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.209510    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gm8nd\""
Jul 15 12:21:18.211845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.211825    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.231856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.231736    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"]
Jul 15 12:21:18.235646 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.235577    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"]
Jul 15 12:21:18.235746 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.235687    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.239688 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.239647    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 15 12:21:18.240807 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.240788    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"]
Jul 15 12:21:18.241040 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.240920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrs74\" (UniqueName: \"kubernetes.io/projected/eeeea4e9-2a94-4af7-85ea-30010b943bb0-kube-api-access-nrs74\") pod \"cluster-samples-operator-65db858894-ls7rx\" (UID: \"eeeea4e9-2a94-4af7-85ea-30010b943bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.241161 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.241161 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d328f198-1825-4446-825a-1bc047a4971c-tmp\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.241161 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtfl7\" (UniqueName: \"kubernetes.io/projected/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-kube-api-access-vtfl7\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.241349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241165    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-snapshots\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.241349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241213    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-tmp\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.241349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241252    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d328f198-1825-4446-825a-1bc047a4971c-klusterlet-config\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.241349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241277    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-stats-auth\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.241349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241302    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-serving-cert\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.241349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241329    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtg7t\" (UniqueName: \"kubernetes.io/projected/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-kube-api-access-xtg7t\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241360    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-default-certificate\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241387    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-service-ca-bundle\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241416    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjhpd\" (UniqueName: \"kubernetes.io/projected/5b2334d3-157e-4349-b2dc-a93e36144b49-kube-api-access-rjhpd\") pod \"volume-data-source-validator-78495597f6-vqh5z\" (UID: \"5b2334d3-157e-4349-b2dc-a93e36144b49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241474    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eeeea4e9-2a94-4af7-85ea-30010b943bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ls7rx\" (UID: \"eeeea4e9-2a94-4af7-85ea-30010b943bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q54c\" (UniqueName: \"kubernetes.io/projected/d328f198-1825-4446-825a-1bc047a4971c-kube-api-access-6q54c\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.241779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-metrics-certs\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.242094 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.241926    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d328f198-1825-4446-825a-1bc047a4971c-tmp\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.245841 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.245799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d328f198-1825-4446-825a-1bc047a4971c-klusterlet-config\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.247033 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.247014    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.248356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.248328    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7p9sh\""
Jul 15 12:21:18.253770 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.253705    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 15 12:21:18.253770 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.253748    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 15 12:21:18.258827 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.258634    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 15 12:21:18.258827 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.258637    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 15 12:21:18.259161 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.259146    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"]
Jul 15 12:21:18.279721 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.279693    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q54c\" (UniqueName: \"kubernetes.io/projected/d328f198-1825-4446-825a-1bc047a4971c-kube-api-access-6q54c\") pod \"klusterlet-addon-workmgr-5547ccdbf7-ffmwj\" (UID: \"d328f198-1825-4446-825a-1bc047a4971c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.310584 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.310551    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"]
Jul 15 12:21:18.342091 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.341923    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-ca\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.342091 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.341966    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-default-certificate\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.342091 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.341992    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-service-ca-bundle\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.343454 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343419    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-service-ca-bundle\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.343592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.342017    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjhpd\" (UniqueName: \"kubernetes.io/projected/5b2334d3-157e-4349-b2dc-a93e36144b49-kube-api-access-rjhpd\") pod \"volume-data-source-validator-78495597f6-vqh5z\" (UID: \"5b2334d3-157e-4349-b2dc-a93e36144b49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"
Jul 15 12:21:18.343592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/db818ac0-1035-4d8c-966b-27351be146ad-additional-service-ca\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.343592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-hub\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.343592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343603    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eeeea4e9-2a94-4af7-85ea-30010b943bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ls7rx\" (UID: \"eeeea4e9-2a94-4af7-85ea-30010b943bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343629    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-metrics-certs\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343651    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb1db474-3a24-48a4-a96d-dae750227727-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6\" (UID: \"eb1db474-3a24-48a4-a96d-dae750227727\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343697    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtwr7\" (UniqueName: \"kubernetes.io/projected/db818ac0-1035-4d8c-966b-27351be146ad-kube-api-access-rtwr7\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrs74\" (UniqueName: \"kubernetes.io/projected/eeeea4e9-2a94-4af7-85ea-30010b943bb0-kube-api-access-nrs74\") pod \"cluster-samples-operator-65db858894-ls7rx\" (UID: \"eeeea4e9-2a94-4af7-85ea-30010b943bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.343800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtfl7\" (UniqueName: \"kubernetes.io/projected/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-kube-api-access-vtfl7\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343838    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-snapshots\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343900    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zxk6\" (UniqueName: \"kubernetes.io/projected/eb1db474-3a24-48a4-a96d-dae750227727-kube-api-access-6zxk6\") pod \"managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6\" (UID: \"eb1db474-3a24-48a4-a96d-dae750227727\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343930    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-tmp\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.343975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-stats-auth\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.344000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-serving-cert\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.344136 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.344030    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtg7t\" (UniqueName: \"kubernetes.io/projected/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-kube-api-access-xtg7t\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.344952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.344517    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"]
Jul 15 12:21:18.345285 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.345246    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-snapshots\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.348304 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.346258    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-tmp\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.348304 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.346428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.348304 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.346648    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.348304 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.348012    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-serving-cert\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.348304 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.348219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/eeeea4e9-2a94-4af7-85ea-30010b943bb0-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-ls7rx\" (UID: \"eeeea4e9-2a94-4af7-85ea-30010b943bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.349668 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.349626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-metrics-certs\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.349781 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.349752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-stats-auth\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.349892 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.349791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-default-certificate\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.352356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.352322    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-dbnzx"]
Jul 15 12:21:18.352581 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.352561    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.356585 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.356366    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.356585 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.356445    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.356843 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.356791    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 15 12:21:18.356991 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.356972    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-49482\""
Jul 15 12:21:18.357951 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.357929    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.358047 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.358027    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 15 12:21:18.371131 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.371095    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 15 12:21:18.371235 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.371189    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-h4ntc\""
Jul 15 12:21:18.371411 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.371393    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:21:18.372152 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.371719    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 15 12:21:18.372652 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.372633    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 15 12:21:18.376863 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.376845    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 15 12:21:18.390016 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.389033    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjhpd\" (UniqueName: \"kubernetes.io/projected/5b2334d3-157e-4349-b2dc-a93e36144b49-kube-api-access-rjhpd\") pod \"volume-data-source-validator-78495597f6-vqh5z\" (UID: \"5b2334d3-157e-4349-b2dc-a93e36144b49\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"
Jul 15 12:21:18.392556 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.392466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtg7t\" (UniqueName: \"kubernetes.io/projected/b5a3b99b-1972-4d70-9f3b-3109e79e1c5c-kube-api-access-xtg7t\") pod \"router-default-595544f54d-bxnpp\" (UID: \"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c\") " pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.399568 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.395259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrs74\" (UniqueName: \"kubernetes.io/projected/eeeea4e9-2a94-4af7-85ea-30010b943bb0-kube-api-access-nrs74\") pod \"cluster-samples-operator-65db858894-ls7rx\" (UID: \"eeeea4e9-2a94-4af7-85ea-30010b943bb0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.399568 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.397152    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"]
Jul 15 12:21:18.403220 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.403170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtfl7\" (UniqueName: \"kubernetes.io/projected/c5eb7b44-4536-4cc3-8ef0-97a8def2a583-kube-api-access-vtfl7\") pod \"insights-operator-68d9dd94cf-zcxwn\" (UID: \"c5eb7b44-4536-4cc3-8ef0-97a8def2a583\") " pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.406720 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.406467    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:18.415490 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.415468    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-dbnzx"]
Jul 15 12:21:18.445071 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445049    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn"
Jul 15 12:21:18.445611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445258    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-hub\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.445611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7jbb\" (UniqueName: \"kubernetes.io/projected/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-kube-api-access-n7jbb\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.445611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445516    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb1db474-3a24-48a4-a96d-dae750227727-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6\" (UID: \"eb1db474-3a24-48a4-a96d-dae750227727\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.445611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445559    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.445611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtwr7\" (UniqueName: \"kubernetes.io/projected/db818ac0-1035-4d8c-966b-27351be146ad-kube-api-access-rtwr7\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445645    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/625c565a-4792-44ae-bc4b-9bf8004d9a52-serving-cert\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445689    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/625c565a-4792-44ae-bc4b-9bf8004d9a52-config\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99jzn\" (UniqueName: \"kubernetes.io/projected/625c565a-4792-44ae-bc4b-9bf8004d9a52-kube-api-access-99jzn\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445744    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/625c565a-4792-44ae-bc4b-9bf8004d9a52-trusted-ca\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445776    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445843    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zxk6\" (UniqueName: \"kubernetes.io/projected/eb1db474-3a24-48a4-a96d-dae750227727-kube-api-access-6zxk6\") pod \"managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6\" (UID: \"eb1db474-3a24-48a4-a96d-dae750227727\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.445914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-ca\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.446289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/db818ac0-1035-4d8c-966b-27351be146ad-additional-service-ca\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.446289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.445967    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.447476 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.447399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/db818ac0-1035-4d8c-966b-27351be146ad-additional-service-ca\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.448928 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.448884    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.449980 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.449932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.450178 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.450144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-ca\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.450301 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.450272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/db818ac0-1035-4d8c-966b-27351be146ad-hub\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.450807 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.450786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/eb1db474-3a24-48a4-a96d-dae750227727-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6\" (UID: \"eb1db474-3a24-48a4-a96d-dae750227727\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.460872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.460852    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:18.479333 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.479249    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"
Jul 15 12:21:18.517230 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.517171    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zxk6\" (UniqueName: \"kubernetes.io/projected/eb1db474-3a24-48a4-a96d-dae750227727-kube-api-access-6zxk6\") pod \"managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6\" (UID: \"eb1db474-3a24-48a4-a96d-dae750227727\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.517813 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.517759    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtwr7\" (UniqueName: \"kubernetes.io/projected/db818ac0-1035-4d8c-966b-27351be146ad-kube-api-access-rtwr7\") pod \"cluster-proxy-proxy-agent-594dd556fb-zgjx7\" (UID: \"db818ac0-1035-4d8c-966b-27351be146ad\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.518461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.518275    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"
Jul 15 12:21:18.546479 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546445    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.546622 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546491    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7jbb\" (UniqueName: \"kubernetes.io/projected/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-kube-api-access-n7jbb\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.546622 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546550    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/625c565a-4792-44ae-bc4b-9bf8004d9a52-serving-cert\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.546622 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/625c565a-4792-44ae-bc4b-9bf8004d9a52-config\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.546622 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546620    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99jzn\" (UniqueName: \"kubernetes.io/projected/625c565a-4792-44ae-bc4b-9bf8004d9a52-kube-api-access-99jzn\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.546816 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546648    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/625c565a-4792-44ae-bc4b-9bf8004d9a52-trusted-ca\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.546816 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.546697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.547567 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.547226    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.547567 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.547498    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/625c565a-4792-44ae-bc4b-9bf8004d9a52-config\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.548400 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.548374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/625c565a-4792-44ae-bc4b-9bf8004d9a52-trusted-ca\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.550289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.550268    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.552323 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.551769    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/625c565a-4792-44ae-bc4b-9bf8004d9a52-serving-cert\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.552323 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.551876    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"
Jul 15 12:21:18.560544 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.560502    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"
Jul 15 12:21:18.571598 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.571347    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7jbb\" (UniqueName: \"kubernetes.io/projected/17d63f31-cbbf-43c0-9ef4-bb82b110f4ce-kube-api-access-n7jbb\") pod \"cluster-monitoring-operator-5f7f494dd4-qnt9h\" (UID: \"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.576098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.576063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99jzn\" (UniqueName: \"kubernetes.io/projected/625c565a-4792-44ae-bc4b-9bf8004d9a52-kube-api-access-99jzn\") pod \"console-operator-7866497494-dbnzx\" (UID: \"625c565a-4792-44ae-bc4b-9bf8004d9a52\") " pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.681160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.681082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"]
Jul 15 12:21:18.692084 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.691886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"
Jul 15 12:21:18.699857 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:18.699680    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd328f198_1825_4446_825a_1bc047a4971c.slice/crio-35ffb0ffa05f6ae5ce1b47a2606df4a29db2e915b5038c496edc8836ff20bffe WatchSource:0}: Error finding container 35ffb0ffa05f6ae5ce1b47a2606df4a29db2e915b5038c496edc8836ff20bffe: Status 404 returned error can't find the container with id 35ffb0ffa05f6ae5ce1b47a2606df4a29db2e915b5038c496edc8836ff20bffe
Jul 15 12:21:18.723722 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.719264    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-zcxwn"]
Jul 15 12:21:18.723722 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.723124    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:18.740235 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:18.739804    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5eb7b44_4536_4cc3_8ef0_97a8def2a583.slice/crio-48685cb84fc057b3dec71c47b1376d82af56073d8f3e5aa784a6301fa06f497e WatchSource:0}: Error finding container 48685cb84fc057b3dec71c47b1376d82af56073d8f3e5aa784a6301fa06f497e: Status 404 returned error can't find the container with id 48685cb84fc057b3dec71c47b1376d82af56073d8f3e5aa784a6301fa06f497e
Jul 15 12:21:18.773856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.773759    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx"]
Jul 15 12:21:18.831853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.827946    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-595544f54d-bxnpp"]
Jul 15 12:21:18.867493 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.867431    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z"]
Jul 15 12:21:18.876647 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.876594    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7"]
Jul 15 12:21:18.878757 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:18.878200    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2334d3_157e_4349_b2dc_a93e36144b49.slice/crio-6694b965108c77e9697efed19700df621606dbd41c251ea49a127cce8dd6a224 WatchSource:0}: Error finding container 6694b965108c77e9697efed19700df621606dbd41c251ea49a127cce8dd6a224: Status 404 returned error can't find the container with id 6694b965108c77e9697efed19700df621606dbd41c251ea49a127cce8dd6a224
Jul 15 12:21:18.890975 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.890888    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjzm8" event={"ID":"877ad258-985f-4f41-ba9b-df2d9e3b4133","Type":"ContainerStarted","Data":"fa93e0908422b7a6e43f19b5045ab806a4c2ccdf3ca5aef6584c6c0ac3d3fb03"}
Jul 15 12:21:18.893815 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.893728    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx" event={"ID":"eeeea4e9-2a94-4af7-85ea-30010b943bb0","Type":"ContainerStarted","Data":"6e7add6464b843bb451babf21ebf0c019504f498f8d6959a070c5a822b3537d1"}
Jul 15 12:21:18.895780 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.895710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn" event={"ID":"c5eb7b44-4536-4cc3-8ef0-97a8def2a583","Type":"ContainerStarted","Data":"48685cb84fc057b3dec71c47b1376d82af56073d8f3e5aa784a6301fa06f497e"}
Jul 15 12:21:18.898844 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.898750    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj" event={"ID":"d328f198-1825-4446-825a-1bc047a4971c","Type":"ContainerStarted","Data":"35ffb0ffa05f6ae5ce1b47a2606df4a29db2e915b5038c496edc8836ff20bffe"}
Jul 15 12:21:18.904015 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.903963    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-595544f54d-bxnpp" event={"ID":"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c","Type":"ContainerStarted","Data":"9712dd5f7ddc9116e5607014bf40189ee7faaf9a6e7865018c776c55aea42abf"}
Jul 15 12:21:18.904582 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.904506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:21:18.943198 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.943166    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h"]
Jul 15 12:21:18.943839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.943488    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mbqqd" podStartSLOduration=67.943469616 podStartE2EDuration="1m7.943469616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:18.941866278 +0000 UTC m=+67.818589283" watchObservedRunningTime="2026-07-15 12:21:18.943469616 +0000 UTC m=+67.820192624"
Jul 15 12:21:18.946242 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:18.946218    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17d63f31_cbbf_43c0_9ef4_bb82b110f4ce.slice/crio-f097da5018957567882c5dde75393e7fac986f5555b7e0ee4038bf256b16f247 WatchSource:0}: Error finding container f097da5018957567882c5dde75393e7fac986f5555b7e0ee4038bf256b16f247: Status 404 returned error can't find the container with id f097da5018957567882c5dde75393e7fac986f5555b7e0ee4038bf256b16f247
Jul 15 12:21:18.951231 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.951211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:21:18.951322 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.951247    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:21:18.953655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.953637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b3b94358-208c-4e52-9d82-b7de8a12e86d-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-m78dh\" (UID: \"b3b94358-208c-4e52-9d82-b7de8a12e86d\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:21:18.954269 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.954228    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"image-registry-6dfb95dbb5-qswrz\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") " pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:21:18.971221 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:18.971082    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-dbnzx"]
Jul 15 12:21:18.976727 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:18.976688    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod625c565a_4792_44ae_bc4b_9bf8004d9a52.slice/crio-e82586ee3e97f7f1d2b6bae49f43f5e9941476b0fe7503c61e83de5c8b84f9c7 WatchSource:0}: Error finding container e82586ee3e97f7f1d2b6bae49f43f5e9941476b0fe7503c61e83de5c8b84f9c7: Status 404 returned error can't find the container with id e82586ee3e97f7f1d2b6bae49f43f5e9941476b0fe7503c61e83de5c8b84f9c7
Jul 15 12:21:19.069634 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.069581    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6"]
Jul 15 12:21:19.072483 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:19.072453    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb1db474_3a24_48a4_a96d_dae750227727.slice/crio-c91198f842245a27f716e7e4cf0f5aa45aa7a0d1546bae1deca3285ce1471cbb WatchSource:0}: Error finding container c91198f842245a27f716e7e4cf0f5aa45aa7a0d1546bae1deca3285ce1471cbb: Status 404 returned error can't find the container with id c91198f842245a27f716e7e4cf0f5aa45aa7a0d1546bae1deca3285ce1471cbb
Jul 15 12:21:19.160870 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.160844    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4bnx5\""
Jul 15 12:21:19.169199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.169175    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:21:19.172033 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.172008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zptxr\""
Jul 15 12:21:19.180169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.180146    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh"
Jul 15 12:21:19.383505 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.383390    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6dfb95dbb5-qswrz"]
Jul 15 12:21:19.427308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.426853    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-m78dh"]
Jul 15 12:21:19.432670 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:19.432634    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b94358_208c_4e52_9d82_b7de8a12e86d.slice/crio-988b1563bf787dd5cc057231295f77f37677176142a2ff292642a2e8b897148c WatchSource:0}: Error finding container 988b1563bf787dd5cc057231295f77f37677176142a2ff292642a2e8b897148c: Status 404 returned error can't find the container with id 988b1563bf787dd5cc057231295f77f37677176142a2ff292642a2e8b897148c
Jul 15 12:21:19.934088 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.933990    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h" event={"ID":"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce","Type":"ContainerStarted","Data":"f097da5018957567882c5dde75393e7fac986f5555b7e0ee4038bf256b16f247"}
Jul 15 12:21:19.944829 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.944770    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7" event={"ID":"db818ac0-1035-4d8c-966b-27351be146ad","Type":"ContainerStarted","Data":"87f5e420bfe44910b5524efc0ca3043ff0d83eacefefcdfd4f59720f1a0ae7d4"}
Jul 15 12:21:19.959194 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.958702    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zjzm8" event={"ID":"877ad258-985f-4f41-ba9b-df2d9e3b4133","Type":"ContainerStarted","Data":"3d224a45d65b0fea64caee281d7f0f3b238eabab897d57c2a56054e7d23aa29b"}
Jul 15 12:21:19.973256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.973225    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dbnzx" event={"ID":"625c565a-4792-44ae-bc4b-9bf8004d9a52","Type":"ContainerStarted","Data":"e82586ee3e97f7f1d2b6bae49f43f5e9941476b0fe7503c61e83de5c8b84f9c7"}
Jul 15 12:21:19.981558 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.981398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6" event={"ID":"eb1db474-3a24-48a4-a96d-dae750227727","Type":"ContainerStarted","Data":"c91198f842245a27f716e7e4cf0f5aa45aa7a0d1546bae1deca3285ce1471cbb"}
Jul 15 12:21:19.983264 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.983186    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zjzm8" podStartSLOduration=68.008646722 podStartE2EDuration="1m8.983168685s" totalImagesPullingTime="974.521963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:19.983040993 +0000 UTC m=+68.859764001" watchObservedRunningTime="2026-07-15 12:21:19.983168685 +0000 UTC m=+68.859891694"
Jul 15 12:21:19.988429 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:19.988349    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z" event={"ID":"5b2334d3-157e-4349-b2dc-a93e36144b49","Type":"ContainerStarted","Data":"6694b965108c77e9697efed19700df621606dbd41c251ea49a127cce8dd6a224"}
Jul 15 12:21:20.012344 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.009281    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-595544f54d-bxnpp" event={"ID":"b5a3b99b-1972-4d70-9f3b-3109e79e1c5c","Type":"ContainerStarted","Data":"25360fa3df2c68751f80739512172ce344e5c92e3cca234264bfa5ab5e8998c2"}
Jul 15 12:21:20.020508 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.020409    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh" event={"ID":"b3b94358-208c-4e52-9d82-b7de8a12e86d","Type":"ContainerStarted","Data":"988b1563bf787dd5cc057231295f77f37677176142a2ff292642a2e8b897148c"}
Jul 15 12:21:20.041129 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.041014    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz" event={"ID":"bdec6837-d575-487c-8e03-4855bf81c163","Type":"ContainerStarted","Data":"a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0"}
Jul 15 12:21:20.041129 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.041058    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz" event={"ID":"bdec6837-d575-487c-8e03-4855bf81c163","Type":"ContainerStarted","Data":"cac4b9fa924e48ba5cdd635ef03e79a10720e4d9688d8e96b7b2d1709ee0e6e1"}
Jul 15 12:21:20.041129 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.041086    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:21:20.093235 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.093026    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-595544f54d-bxnpp" podStartSLOduration=2.093001577 podStartE2EDuration="2.093001577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:20.053013321 +0000 UTC m=+68.929736328" watchObservedRunningTime="2026-07-15 12:21:20.093001577 +0000 UTC m=+68.969724584"
Jul 15 12:21:20.462846 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.462759    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:20.466084 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.465454    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:20.494804 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:20.493370    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz" podStartSLOduration=69.493349025 podStartE2EDuration="1m9.493349025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:20.093683538 +0000 UTC m=+68.970406543" watchObservedRunningTime="2026-07-15 12:21:20.493349025 +0000 UTC m=+69.370072062"
Jul 15 12:21:21.055734 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:21.055459    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:21.057485 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:21.056806    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-595544f54d-bxnpp"
Jul 15 12:21:30.096822 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.095861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj" event={"ID":"d328f198-1825-4446-825a-1bc047a4971c","Type":"ContainerStarted","Data":"037f91f33a1c2fdbf34487969061554139aeaec10d716be33c8afab39ecd9f76"}
Jul 15 12:21:30.096822 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.095936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:30.101184 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.100335    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj"
Jul 15 12:21:30.101463 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.101416    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h" event={"ID":"17d63f31-cbbf-43c0-9ef4-bb82b110f4ce","Type":"ContainerStarted","Data":"098101611e0aa5110ccc82f5c6fbd83490c30914f9cd2eb956a17222bdbb7b60"}
Jul 15 12:21:30.105792 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.105674    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7" event={"ID":"db818ac0-1035-4d8c-966b-27351be146ad","Type":"ContainerStarted","Data":"31a241fa6872015264ae994d52c58ce70c617852c66802de09b0410531c0c7f8"}
Jul 15 12:21:30.105792 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.105710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7" event={"ID":"db818ac0-1035-4d8c-966b-27351be146ad","Type":"ContainerStarted","Data":"e33eae29bbc6b4c67e9fdf5ae1998fa4d470be9f83f44237289ba8f63db80c12"}
Jul 15 12:21:30.105792 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.105725    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7" event={"ID":"db818ac0-1035-4d8c-966b-27351be146ad","Type":"ContainerStarted","Data":"b2e2ce5874078822ea03d5fdd679baeb934acd68611d684d10dd2cf057278ab2"}
Jul 15 12:21:30.107818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.107771    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-dbnzx" event={"ID":"625c565a-4792-44ae-bc4b-9bf8004d9a52","Type":"ContainerStarted","Data":"be7b8330ad56253d3496b7c0293f69c5d133594dc57ba1e793805bcc98a98d4d"}
Jul 15 12:21:30.108001 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.107955    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:30.111705 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.111677    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z" event={"ID":"5b2334d3-157e-4349-b2dc-a93e36144b49","Type":"ContainerStarted","Data":"5b8daa77125d04d4662d16fd7307933cc3a3d2c0789e27df39fffeb4049aa08b"}
Jul 15 12:21:30.114324 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.114299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx" event={"ID":"eeeea4e9-2a94-4af7-85ea-30010b943bb0","Type":"ContainerStarted","Data":"35a2abead386877ffbd7692bc2860121b50e583e7213bbc830795d559a0b2e5d"}
Jul 15 12:21:30.114408 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.114360    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx" event={"ID":"eeeea4e9-2a94-4af7-85ea-30010b943bb0","Type":"ContainerStarted","Data":"5eb3691537f6cc31e43dccf5a286eb94fa1b97ff7c7f38b5c23056f24342bd30"}
Jul 15 12:21:30.116269 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.116245    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn" event={"ID":"c5eb7b44-4536-4cc3-8ef0-97a8def2a583","Type":"ContainerStarted","Data":"2aee759a16d36c1925317e7ae9e0c50f2440478ca6f0911b55666a921ff35871"}
Jul 15 12:21:30.122168 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.122115    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5547ccdbf7-ffmwj" podStartSLOduration=1.255974951 podStartE2EDuration="12.122097878s" totalImagesPullingTime="10.866122927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.118995373 +0000 UTC m=+78.995718379" watchObservedRunningTime="2026-07-15 12:21:30.122097878 +0000 UTC m=+78.998820884"
Jul 15 12:21:30.126511 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.126485    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh" event={"ID":"b3b94358-208c-4e52-9d82-b7de8a12e86d","Type":"ContainerStarted","Data":"1a7f0563cde622f201728461335fbb17d12c33261afb0960260f4b128d358039"}
Jul 15 12:21:30.132497 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.132389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7kfpt" event={"ID":"ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6","Type":"ContainerStarted","Data":"4dc36f80942450fb28fe5dedb5c80a1496fd47768f6e6119011a0ed0359f4558"}
Jul 15 12:21:30.148020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.147964    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-dbnzx" podStartSLOduration=1.579345767 podStartE2EDuration="12.147951277s" totalImagesPullingTime="10.56860551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.146985081 +0000 UTC m=+79.023708100" watchObservedRunningTime="2026-07-15 12:21:30.147951277 +0000 UTC m=+79.024674265"
Jul 15 12:21:30.227748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.227670    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-ls7rx" podStartSLOduration=1.554836258 podStartE2EDuration="12.227654676s" totalImagesPullingTime="10.672818418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.22711799 +0000 UTC m=+79.103841045" watchObservedRunningTime="2026-07-15 12:21:30.227654676 +0000 UTC m=+79.104377683"
Jul 15 12:21:30.228007 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.227835    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-vqh5z" podStartSLOduration=1.5813195869999999 podStartE2EDuration="12.227827948s" totalImagesPullingTime="10.646508361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.20066537 +0000 UTC m=+79.077388378" watchObservedRunningTime="2026-07-15 12:21:30.227827948 +0000 UTC m=+79.104550954"
Jul 15 12:21:30.255374 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.255247    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-zcxwn" podStartSLOduration=1.452650828 podStartE2EDuration="12.25522725s" totalImagesPullingTime="10.802576422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.253729441 +0000 UTC m=+79.130452447" watchObservedRunningTime="2026-07-15 12:21:30.25522725 +0000 UTC m=+79.131950257"
Jul 15 12:21:30.291800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.290153    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-qnt9h" podStartSLOduration=1.693541878 podStartE2EDuration="12.290134685s" totalImagesPullingTime="10.596592807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.289755761 +0000 UTC m=+79.166478769" watchObservedRunningTime="2026-07-15 12:21:30.290134685 +0000 UTC m=+79.166857692"
Jul 15 12:21:30.322571 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.322488    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-594dd556fb-zgjx7" podStartSLOduration=1.644934407 podStartE2EDuration="12.322469768s" totalImagesPullingTime="10.677535361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.321594345 +0000 UTC m=+79.198317353" watchObservedRunningTime="2026-07-15 12:21:30.322469768 +0000 UTC m=+79.199192775"
Jul 15 12:21:30.344862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.344800    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7kfpt" podStartSLOduration=66.336911351 podStartE2EDuration="1m18.34478127s" totalImagesPullingTime="12.007869919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.343804378 +0000 UTC m=+79.220527385" watchObservedRunningTime="2026-07-15 12:21:30.34478127 +0000 UTC m=+79.221504276"
Jul 15 12:21:30.374207 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.374157    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-m78dh" podStartSLOduration=38.265765312 podStartE2EDuration="48.37414256s" totalImagesPullingTime="10.108377248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:30.373258712 +0000 UTC m=+79.249981719" watchObservedRunningTime="2026-07-15 12:21:30.37414256 +0000 UTC m=+79.250865567"
Jul 15 12:21:30.571001 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:30.570888    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-dbnzx"
Jul 15 12:21:31.043696 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.043663    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mtxtm"]
Jul 15 12:21:31.047335 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.047310    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.052035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.052013    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 15 12:21:31.052035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.052012    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 15 12:21:31.052283 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.052269    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8wtrb\""
Jul 15 12:21:31.086556 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.086506    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mtxtm"]
Jul 15 12:21:31.137495 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.137421    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6" event={"ID":"eb1db474-3a24-48a4-a96d-dae750227727","Type":"ContainerStarted","Data":"74900e9995ed8e0b0ff88e9cf4db84e9375d5aca1ab1bad947b7b9451a71bfe3"}
Jul 15 12:21:31.160032 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.159987    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f8b7464bd-k9kt6" podStartSLOduration=1.346889752 podStartE2EDuration="13.159973566s" totalImagesPullingTime="11.813083814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:21:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:31.158878542 +0000 UTC m=+80.035601547" watchObservedRunningTime="2026-07-15 12:21:31.159973566 +0000 UTC m=+80.036696568"
Jul 15 12:21:31.173152 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.173127    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.173263 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.173169    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-crio-socket\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.173263 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.173185    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-data-volume\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.173263 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.173207    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bdn2\" (UniqueName: \"kubernetes.io/projected/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-api-access-4bdn2\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.173263 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.173237    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.274077 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.274236 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.274171    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:21:31.274311 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.274246    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm podName:b555f300-48b7-4ff5-ba2b-4c45bec3e8df nodeName:}" failed. No retries permitted until 2026-07-15 12:21:31.774225287 +0000 UTC m=+80.650948292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtxtm" (UID: "b555f300-48b7-4ff5-ba2b-4c45bec3e8df") : configmap "kube-rbac-proxy" not found
Jul 15 12:21:31.274400 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-crio-socket\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.274881 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274514    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-data-volume\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.274881 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-crio-socket\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.274881 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bdn2\" (UniqueName: \"kubernetes.io/projected/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-api-access-4bdn2\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.274881 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.275160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.274928    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-data-volume\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.275160 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.274952    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:21:31.275160 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.275010    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls podName:b555f300-48b7-4ff5-ba2b-4c45bec3e8df nodeName:}" failed. No retries permitted until 2026-07-15 12:21:31.774992946 +0000 UTC m=+80.651715937 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtxtm" (UID: "b555f300-48b7-4ff5-ba2b-4c45bec3e8df") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:21:31.299065 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.299040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bdn2\" (UniqueName: \"kubernetes.io/projected/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-api-access-4bdn2\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.779306 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.779276    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.779462 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:31.779334    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:31.779462 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.779411    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:21:31.779462 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.779459    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm podName:b555f300-48b7-4ff5-ba2b-4c45bec3e8df nodeName:}" failed. No retries permitted until 2026-07-15 12:21:32.779446164 +0000 UTC m=+81.656169148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtxtm" (UID: "b555f300-48b7-4ff5-ba2b-4c45bec3e8df") : configmap "kube-rbac-proxy" not found
Jul 15 12:21:31.779600 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.779409    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:21:31.779600 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:31.779555    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls podName:b555f300-48b7-4ff5-ba2b-4c45bec3e8df nodeName:}" failed. No retries permitted until 2026-07-15 12:21:32.77952403 +0000 UTC m=+81.656247014 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtxtm" (UID: "b555f300-48b7-4ff5-ba2b-4c45bec3e8df") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:21:32.457176 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:32.457148    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zgxhc_95aa8e11-cade-4b79-bd27-10d2459910a4/dns/0.log"
Jul 15 12:21:32.650997 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:32.650966    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zgxhc_95aa8e11-cade-4b79-bd27-10d2459910a4/kube-rbac-proxy/0.log"
Jul 15 12:21:32.786129 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:32.786056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:32.786129 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:32.786112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:32.786340 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:32.786186    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 15 12:21:32.786340 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:32.786227    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 15 12:21:32.786340 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:32.786252    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm podName:b555f300-48b7-4ff5-ba2b-4c45bec3e8df nodeName:}" failed. No retries permitted until 2026-07-15 12:21:34.786236573 +0000 UTC m=+83.662959564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mtxtm" (UID: "b555f300-48b7-4ff5-ba2b-4c45bec3e8df") : configmap "kube-rbac-proxy" not found
Jul 15 12:21:32.786340 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:21:32.786287    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls podName:b555f300-48b7-4ff5-ba2b-4c45bec3e8df nodeName:}" failed. No retries permitted until 2026-07-15 12:21:34.786270361 +0000 UTC m=+83.662993348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mtxtm" (UID: "b555f300-48b7-4ff5-ba2b-4c45bec3e8df") : secret "insights-runtime-extractor-tls" not found
Jul 15 12:21:33.053249 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:33.053173    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hgvm9_3d6d2951-2153-434e-b87f-bff9e54fb088/dns-node-resolver/0.log"
Jul 15 12:21:33.649217 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:33.649191    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6dfb95dbb5-qswrz_bdec6837-d575-487c-8e03-4855bf81c163/registry/0.log"
Jul 15 12:21:34.056817 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.056741    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fjhkd_15a5102c-c886-4f37-ba0c-2bb0f653fa09/node-ca/0.log"
Jul 15 12:21:34.449363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.449339    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-595544f54d-bxnpp_b5a3b99b-1972-4d70-9f3b-3109e79e1c5c/router/0.log"
Jul 15 12:21:34.648929 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.648905    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pwmnc_a560a9c7-677a-4ad2-9b65-1e15843b171d/serve-healthcheck-canary/0.log"
Jul 15 12:21:34.802663 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.802584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:34.802663 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.802658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:34.803154 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.803131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:34.804779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.804759    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b555f300-48b7-4ff5-ba2b-4c45bec3e8df-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mtxtm\" (UID: \"b555f300-48b7-4ff5-ba2b-4c45bec3e8df\") " pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:34.957970 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:34.957940    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mtxtm"
Jul 15 12:21:35.094348 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:35.094274    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mtxtm"]
Jul 15 12:21:35.097586 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:35.097561    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb555f300_48b7_4ff5_ba2b_4c45bec3e8df.slice/crio-b49c5c949b5ce59ce1d5edd0739a3bda26ecb13d4be46f51a5a8153543c24a46 WatchSource:0}: Error finding container b49c5c949b5ce59ce1d5edd0739a3bda26ecb13d4be46f51a5a8153543c24a46: Status 404 returned error can't find the container with id b49c5c949b5ce59ce1d5edd0739a3bda26ecb13d4be46f51a5a8153543c24a46
Jul 15 12:21:35.153476 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:35.153438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtxtm" event={"ID":"b555f300-48b7-4ff5-ba2b-4c45bec3e8df","Type":"ContainerStarted","Data":"b49c5c949b5ce59ce1d5edd0739a3bda26ecb13d4be46f51a5a8153543c24a46"}
Jul 15 12:21:40.177016 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:40.176982    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtxtm" event={"ID":"b555f300-48b7-4ff5-ba2b-4c45bec3e8df","Type":"ContainerStarted","Data":"3d177628b8715c14b77cd27021fb6280a38039d996bd3f40063ed1fc6a7bdcc7"}
Jul 15 12:21:41.059981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:41.059953    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:21:41.866442 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:41.866404    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dfb95dbb5-qswrz"]
Jul 15 12:21:44.198951 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:44.198914    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtxtm" event={"ID":"b555f300-48b7-4ff5-ba2b-4c45bec3e8df","Type":"ContainerStarted","Data":"add0dd4bd9c7c329f325e2037305b9f4579517b90d44957707df302717de55da"}
Jul 15 12:21:47.212575 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:47.212523    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mtxtm" event={"ID":"b555f300-48b7-4ff5-ba2b-4c45bec3e8df","Type":"ContainerStarted","Data":"d5436c8a424a97e290a04b5decf6a8d87b98183f1e3e73b8996259b0f274634f"}
Jul 15 12:21:47.241373 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:47.241327    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mtxtm" podStartSLOduration=4.577836383 podStartE2EDuration="16.241313388s" totalImagesPullingTime="11.663477005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 12:21:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:47.239506151 +0000 UTC m=+96.116229157" watchObservedRunningTime="2026-07-15 12:21:47.241313388 +0000 UTC m=+96.118036393"
Jul 15 12:21:50.042659 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:50.042622    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mbqqd"
Jul 15 12:21:55.780317 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.780282    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zkfv5"]
Jul 15 12:21:55.848850 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.848820    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.852900 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.852870    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 15 12:21:55.853696 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.853675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g745z\""
Jul 15 12:21:55.853917 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.853897    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 15 12:21:55.854028 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.853938    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 15 12:21:55.854028 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.853969    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 15 12:21:55.879321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2ba17b-d828-4945-92f6-556aac9d4ac2-metrics-client-ca\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879478 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879332    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-accelerators-collector-config\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879478 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879374    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqq5t\" (UniqueName: \"kubernetes.io/projected/1c2ba17b-d828-4945-92f6-556aac9d4ac2-kube-api-access-kqq5t\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879478 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-tls\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879478 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879429    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-root\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879649 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879478    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-sys\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879649 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879506    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-wtmp\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879649 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.879754 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.879646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-textfile\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980482 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980448    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-textfile\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980497    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2ba17b-d828-4945-92f6-556aac9d4ac2-metrics-client-ca\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-accelerators-collector-config\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980567    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqq5t\" (UniqueName: \"kubernetes.io/projected/1c2ba17b-d828-4945-92f6-556aac9d4ac2-kube-api-access-kqq5t\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-tls\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980608    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-root\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-sys\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.980687 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980681    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-wtmp\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980948    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-root\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980953    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-textfile\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.980964    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-sys\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981275 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.981071    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-wtmp\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981275 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.981128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1c2ba17b-d828-4945-92f6-556aac9d4ac2-metrics-client-ca\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.981275 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.981147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-accelerators-collector-config\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.983164 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.983135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-tls\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.983270 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.983194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1c2ba17b-d828-4945-92f6-556aac9d4ac2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:55.992086 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:55.992061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqq5t\" (UniqueName: \"kubernetes.io/projected/1c2ba17b-d828-4945-92f6-556aac9d4ac2-kube-api-access-kqq5t\") pod \"node-exporter-zkfv5\" (UID: \"1c2ba17b-d828-4945-92f6-556aac9d4ac2\") " pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:56.159802 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:56.159769    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zkfv5"
Jul 15 12:21:56.198243 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:21:56.198213    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c2ba17b_d828_4945_92f6_556aac9d4ac2.slice/crio-7fee1d2735eea93818e38a4bd80faeb334253ae182cbad646870ec0c5d4d28c2 WatchSource:0}: Error finding container 7fee1d2735eea93818e38a4bd80faeb334253ae182cbad646870ec0c5d4d28c2: Status 404 returned error can't find the container with id 7fee1d2735eea93818e38a4bd80faeb334253ae182cbad646870ec0c5d4d28c2
Jul 15 12:21:56.247748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:56.247712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkfv5" event={"ID":"1c2ba17b-d828-4945-92f6-556aac9d4ac2","Type":"ContainerStarted","Data":"7fee1d2735eea93818e38a4bd80faeb334253ae182cbad646870ec0c5d4d28c2"}
Jul 15 12:21:58.257999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:58.257965    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c2ba17b-d828-4945-92f6-556aac9d4ac2" containerID="cb9c839767d63d969d7662014778438a2efc6991e52d07e27bb15ca923201383" exitCode=0
Jul 15 12:21:58.258452 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:58.258039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkfv5" event={"ID":"1c2ba17b-d828-4945-92f6-556aac9d4ac2","Type":"ContainerDied","Data":"cb9c839767d63d969d7662014778438a2efc6991e52d07e27bb15ca923201383"}
Jul 15 12:21:59.264071 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:59.264035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkfv5" event={"ID":"1c2ba17b-d828-4945-92f6-556aac9d4ac2","Type":"ContainerStarted","Data":"675cbc817f7ea0eb0926dd81bb3def3fe457c02f5662db32afd9315e2b804561"}
Jul 15 12:21:59.264414 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:59.264087    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zkfv5" event={"ID":"1c2ba17b-d828-4945-92f6-556aac9d4ac2","Type":"ContainerStarted","Data":"899f5a1c50c4435fceae10db973536f8ed3fd76ee6691e2bcf8c5bd025574601"}
Jul 15 12:21:59.291928 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:21:59.291881    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zkfv5" podStartSLOduration=2.789565561 podStartE2EDuration="4.291866843s" totalImagesPullingTime="1.438509369s" totalInitContainerRuntime="63.791913ms" isStatefulPod=true podCreationTimestamp="2026-07-15 12:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:21:59.289751737 +0000 UTC m=+108.166474756" watchObservedRunningTime="2026-07-15 12:21:59.291866843 +0000 UTC m=+108.168589848"
Jul 15 12:22:06.891628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:06.891564    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz" podUID="bdec6837-d575-487c-8e03-4855bf81c163" containerName="registry" containerID="cri-o://a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0" gracePeriod=30
Jul 15 12:22:07.173541 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.173506    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:22:07.271778 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271746    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-registry-certificates\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271778 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271777    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271810    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-installation-pull-secrets\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271851    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g95rk\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-kube-api-access-g95rk\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-bound-sa-token\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271907    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdec6837-d575-487c-8e03-4855bf81c163-ca-trust-extracted\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271926    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-trusted-ca\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.271998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.271950    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-image-registry-private-configuration\") pod \"bdec6837-d575-487c-8e03-4855bf81c163\" (UID: \"bdec6837-d575-487c-8e03-4855bf81c163\") "
Jul 15 12:22:07.272700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.272599    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:22:07.272700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.272643    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:22:07.274361 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.274310    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:22:07.274511 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.274476    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-kube-api-access-g95rk" (OuterVolumeSpecName: "kube-api-access-g95rk") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "kube-api-access-g95rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:22:07.274600 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.274544    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:22:07.274662 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.274604    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:22:07.274715 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.274669    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:22:07.279589 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.279562    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdec6837-d575-487c-8e03-4855bf81c163-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bdec6837-d575-487c-8e03-4855bf81c163" (UID: "bdec6837-d575-487c-8e03-4855bf81c163"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:22:07.295569 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.295519    2576 generic.go:356] "Generic (PLEG): container finished" podID="bdec6837-d575-487c-8e03-4855bf81c163" containerID="a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0" exitCode=0
Jul 15 12:22:07.295661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.295560    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz" event={"ID":"bdec6837-d575-487c-8e03-4855bf81c163","Type":"ContainerDied","Data":"a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0"}
Jul 15 12:22:07.295661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.295592    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz"
Jul 15 12:22:07.295661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.295620    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6dfb95dbb5-qswrz" event={"ID":"bdec6837-d575-487c-8e03-4855bf81c163","Type":"ContainerDied","Data":"cac4b9fa924e48ba5cdd635ef03e79a10720e4d9688d8e96b7b2d1709ee0e6e1"}
Jul 15 12:22:07.295661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.295647    2576 scope.go:117] "RemoveContainer" containerID="a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0"
Jul 15 12:22:07.306598 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.306582    2576 scope.go:117] "RemoveContainer" containerID="a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0"
Jul 15 12:22:07.306888 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:22:07.306862    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0\": container with ID starting with a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0 not found: ID does not exist" containerID="a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0"
Jul 15 12:22:07.306935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.306900    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0"} err="failed to get container status \"a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0\": rpc error: code = NotFound desc = could not find container \"a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0\": container with ID starting with a85eac753e3259adfe0377e7cdaa3e97feaa1da1e55c80910805be18db25adf0 not found: ID does not exist"
Jul 15 12:22:07.325922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.325895    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6dfb95dbb5-qswrz"]
Jul 15 12:22:07.329236 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.329218    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6dfb95dbb5-qswrz"]
Jul 15 12:22:07.373287 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373267    2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bdec6837-d575-487c-8e03-4855bf81c163-ca-trust-extracted\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373287 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373285    2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-trusted-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373295    2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-image-registry-private-configuration\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373305    2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bdec6837-d575-487c-8e03-4855bf81c163-registry-certificates\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373313    2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-registry-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373322    2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bdec6837-d575-487c-8e03-4855bf81c163-installation-pull-secrets\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373330    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g95rk\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-kube-api-access-g95rk\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.373407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.373338    2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bdec6837-d575-487c-8e03-4855bf81c163-bound-sa-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:22:07.571734 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:07.571659    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdec6837-d575-487c-8e03-4855bf81c163" path="/var/lib/kubelet/pods/bdec6837-d575-487c-8e03-4855bf81c163/volumes"
Jul 15 12:22:09.304800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:09.304769    2576 generic.go:356] "Generic (PLEG): container finished" podID="c74f7ff8-ae45-4bf9-ad15-59abca5df8b2" containerID="80fb8f117b2c5b707493f6560bcfb083ef68b1ab5889d680400c39a7481a391c" exitCode=0
Jul 15 12:22:09.305256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:09.304843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh" event={"ID":"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2","Type":"ContainerDied","Data":"80fb8f117b2c5b707493f6560bcfb083ef68b1ab5889d680400c39a7481a391c"}
Jul 15 12:22:09.305256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:09.305234    2576 scope.go:117] "RemoveContainer" containerID="80fb8f117b2c5b707493f6560bcfb083ef68b1ab5889d680400c39a7481a391c"
Jul 15 12:22:10.309933 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:10.309895    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-fmzhh" event={"ID":"c74f7ff8-ae45-4bf9-ad15-59abca5df8b2","Type":"ContainerStarted","Data":"27e4cae1f566cb476d493754c2e2ee3ba27b659cf095ed3aebaa912db1e04be4"}
Jul 15 12:22:24.363391 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:24.363309    2576 generic.go:356] "Generic (PLEG): container finished" podID="41151eeb-4c7c-4839-95ed-2f54de1b0360" containerID="335e5e330245e2cfff665c8509d0c2b97ca0045b419c66b1350d2c9d0b36de30" exitCode=0
Jul 15 12:22:24.363391 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:24.363379    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t" event={"ID":"41151eeb-4c7c-4839-95ed-2f54de1b0360","Type":"ContainerDied","Data":"335e5e330245e2cfff665c8509d0c2b97ca0045b419c66b1350d2c9d0b36de30"}
Jul 15 12:22:24.363836 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:24.363707    2576 scope.go:117] "RemoveContainer" containerID="335e5e330245e2cfff665c8509d0c2b97ca0045b419c66b1350d2c9d0b36de30"
Jul 15 12:22:25.368909 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:22:25.368876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-tsd4t" event={"ID":"41151eeb-4c7c-4839-95ed-2f54de1b0360","Type":"ContainerStarted","Data":"fe2f8415de8d0d3bbf0a91af372af142493a60837fdd60286d7fea2cf33490ba"}
Jul 15 12:25:11.443477 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:11.443450    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:25:11.443981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:11.443893    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:25:11.447036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:11.447019    2576 kubelet.go:1648] "Image garbage collection succeeded"
Jul 15 12:25:37.497566 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.497520    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"]
Jul 15 12:25:37.499920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.497841    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdec6837-d575-487c-8e03-4855bf81c163" containerName="registry"
Jul 15 12:25:37.499920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.497853    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdec6837-d575-487c-8e03-4855bf81c163" containerName="registry"
Jul 15 12:25:37.499920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.497921    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdec6837-d575-487c-8e03-4855bf81c163" containerName="registry"
Jul 15 12:25:37.500803 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.500787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.503676 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.503652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-manifests\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.503801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.503681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.503801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.503701    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.503801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.503734    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nrk8\" (UniqueName: \"kubernetes.io/projected/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-kube-api-access-9nrk8\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.503944 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.503819    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-tmp\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.505669 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.505652    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 15 12:25:37.508865 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.508846    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 15 12:25:37.510965 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.510947    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-657ml\""
Jul 15 12:25:37.511052 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.510986    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 15 12:25:37.511052 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.511029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 15 12:25:37.553155 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.553118    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"]
Jul 15 12:25:37.605026 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.604973    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-manifests\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.605026 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.605015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.605293 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.605044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.605293 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.605080    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nrk8\" (UniqueName: \"kubernetes.io/projected/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-kube-api-access-9nrk8\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.605293 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.605112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-tmp\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.605752 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.605715    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-manifests\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.607872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.607841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.607964 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.607940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-tmp\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.608276 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.608255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.619486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.619460    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nrk8\" (UniqueName: \"kubernetes.io/projected/bb2ac1c9-0b49-4af4-b9f9-90a90f46349a-kube-api-access-9nrk8\") pod \"opendatahub-operator-controller-manager-5f56ddd455-b7lvd\" (UID: \"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a\") " pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.810973 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.810886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:37.972585 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.972552    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"]
Jul 15 12:25:37.976699 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:25:37.976672    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2ac1c9_0b49_4af4_b9f9_90a90f46349a.slice/crio-466729da94fb91f9afb8367f509da6f6f37a27790b9d7e460f5d1420a16725ed WatchSource:0}: Error finding container 466729da94fb91f9afb8367f509da6f6f37a27790b9d7e460f5d1420a16725ed: Status 404 returned error can't find the container with id 466729da94fb91f9afb8367f509da6f6f37a27790b9d7e460f5d1420a16725ed
Jul 15 12:25:37.978628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:37.978611    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:25:38.070648 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:38.070578    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd" event={"ID":"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a","Type":"ContainerStarted","Data":"466729da94fb91f9afb8367f509da6f6f37a27790b9d7e460f5d1420a16725ed"}
Jul 15 12:25:41.086134 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:41.086099    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd" event={"ID":"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a","Type":"ContainerStarted","Data":"84de1c0e220f6fd8c14ecde67346d490ed25656d645549d88c2bf8b1908276ee"}
Jul 15 12:25:42.091652 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:42.091612    2576 generic.go:356] "Generic (PLEG): container finished" podID="bb2ac1c9-0b49-4af4-b9f9-90a90f46349a" containerID="84de1c0e220f6fd8c14ecde67346d490ed25656d645549d88c2bf8b1908276ee" exitCode=0
Jul 15 12:25:42.091992 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:42.091686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd" event={"ID":"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a","Type":"ContainerDied","Data":"84de1c0e220f6fd8c14ecde67346d490ed25656d645549d88c2bf8b1908276ee"}
Jul 15 12:25:43.097927 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:43.097891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd" event={"ID":"bb2ac1c9-0b49-4af4-b9f9-90a90f46349a","Type":"ContainerStarted","Data":"971c6149d23007a04f46e1c2eaa554fab5fb0ee695f33cf8fb4f0b2536edf988"}
Jul 15 12:25:43.098311 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:43.098028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:25:43.139722 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:43.139667    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd" podStartSLOduration=1.759913726 podStartE2EDuration="6.139650689s" totalImagesPullingTime="3.326748032s" totalInitContainerRuntime="1.052988931s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:25:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:25:43.13667609 +0000 UTC m=+332.013399110" watchObservedRunningTime="2026-07-15 12:25:43.139650689 +0000 UTC m=+332.016373694"
Jul 15 12:25:54.104621 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:25:54.104589    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f56ddd455-b7lvd"
Jul 15 12:26:06.692461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.692386    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"]
Jul 15 12:26:06.695575 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.695559    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.702373 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.701801    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 15 12:26:06.704378 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.704356    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 15 12:26:06.704818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.704804    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 15 12:26:06.704872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.704820    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 15 12:26:06.704908 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.704805    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k2zm9\""
Jul 15 12:26:06.706029 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.706016    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 15 12:26:06.733795 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.733773    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"]
Jul 15 12:26:06.853525 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.853491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-cert\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.853712 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.853557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-manager-config\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.853712 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.853594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sl4w\" (UniqueName: \"kubernetes.io/projected/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-kube-api-access-8sl4w\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.853712 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.853612    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-metrics-cert\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.955148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.955052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-cert\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.955148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.955095    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-manager-config\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.955148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.955121    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sl4w\" (UniqueName: \"kubernetes.io/projected/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-kube-api-access-8sl4w\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.955148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.955139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-metrics-cert\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.955829 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.955806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-manager-config\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.958321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.958287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-cert\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.958891 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.958870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-metrics-cert\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:06.978274 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:06.978255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sl4w\" (UniqueName: \"kubernetes.io/projected/f29a74b3-777c-4c23-8fd7-339bcb8f2df1-kube-api-access-8sl4w\") pod \"lws-controller-manager-595c7f4b4c-5nqsk\" (UID: \"f29a74b3-777c-4c23-8fd7-339bcb8f2df1\") " pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:07.005259 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:07.005233    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:07.224905 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:07.224886    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"]
Jul 15 12:26:07.226525 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:26:07.226498    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29a74b3_777c_4c23_8fd7_339bcb8f2df1.slice/crio-182bd582fd7643ef574fc1f2f865935a0153be59072020c9d612bd6445c4192c WatchSource:0}: Error finding container 182bd582fd7643ef574fc1f2f865935a0153be59072020c9d612bd6445c4192c: Status 404 returned error can't find the container with id 182bd582fd7643ef574fc1f2f865935a0153be59072020c9d612bd6445c4192c
Jul 15 12:26:08.210055 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:08.210012    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk" event={"ID":"f29a74b3-777c-4c23-8fd7-339bcb8f2df1","Type":"ContainerStarted","Data":"182bd582fd7643ef574fc1f2f865935a0153be59072020c9d612bd6445c4192c"}
Jul 15 12:26:10.221339 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.221309    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk" event={"ID":"f29a74b3-777c-4c23-8fd7-339bcb8f2df1","Type":"ContainerStarted","Data":"a87becce5733824a8aa9faf5ed7a05bc203216593c297460bca3632142fc849c"}
Jul 15 12:26:10.221707 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.221372    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:10.253221 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.253163    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk" podStartSLOduration=1.8147175610000001 podStartE2EDuration="4.253148347s" totalImagesPullingTime="2.438430786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:10.252146458 +0000 UTC m=+359.128869464" watchObservedRunningTime="2026-07-15 12:26:10.253148347 +0000 UTC m=+359.129871353"
Jul 15 12:26:10.875170 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.875134    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"]
Jul 15 12:26:10.878885 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.878859    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:10.883915 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.883826    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 15 12:26:10.883915 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.883903    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-sz8t5\""
Jul 15 12:26:10.907407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.907376    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"]
Jul 15 12:26:10.989663 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.989610    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57dc7b45-6bde-44ce-ab43-7f38c5a19fae-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-679db\" (UID: \"57dc7b45-6bde-44ce-ab43-7f38c5a19fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:10.989829 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:10.989695    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67pj\" (UniqueName: \"kubernetes.io/projected/57dc7b45-6bde-44ce-ab43-7f38c5a19fae-kube-api-access-k67pj\") pod \"llmisvc-controller-manager-777c6bd7d5-679db\" (UID: \"57dc7b45-6bde-44ce-ab43-7f38c5a19fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:11.090678 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:11.090647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67pj\" (UniqueName: \"kubernetes.io/projected/57dc7b45-6bde-44ce-ab43-7f38c5a19fae-kube-api-access-k67pj\") pod \"llmisvc-controller-manager-777c6bd7d5-679db\" (UID: \"57dc7b45-6bde-44ce-ab43-7f38c5a19fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:11.090835 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:11.090739    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57dc7b45-6bde-44ce-ab43-7f38c5a19fae-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-679db\" (UID: \"57dc7b45-6bde-44ce-ab43-7f38c5a19fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:11.093049 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:11.093031    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57dc7b45-6bde-44ce-ab43-7f38c5a19fae-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-679db\" (UID: \"57dc7b45-6bde-44ce-ab43-7f38c5a19fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:11.108196 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:11.108162    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67pj\" (UniqueName: \"kubernetes.io/projected/57dc7b45-6bde-44ce-ab43-7f38c5a19fae-kube-api-access-k67pj\") pod \"llmisvc-controller-manager-777c6bd7d5-679db\" (UID: \"57dc7b45-6bde-44ce-ab43-7f38c5a19fae\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:11.191768 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:11.191736    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:11.371493 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:11.371459    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"]
Jul 15 12:26:11.375265 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:26:11.375240    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod57dc7b45_6bde_44ce_ab43_7f38c5a19fae.slice/crio-b443f280e96e6daaced198b79d39bbc3b3a1eedf6bebbf305481966bcbce8846 WatchSource:0}: Error finding container b443f280e96e6daaced198b79d39bbc3b3a1eedf6bebbf305481966bcbce8846: Status 404 returned error can't find the container with id b443f280e96e6daaced198b79d39bbc3b3a1eedf6bebbf305481966bcbce8846
Jul 15 12:26:12.230404 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:12.230368    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db" event={"ID":"57dc7b45-6bde-44ce-ab43-7f38c5a19fae","Type":"ContainerStarted","Data":"b443f280e96e6daaced198b79d39bbc3b3a1eedf6bebbf305481966bcbce8846"}
Jul 15 12:26:15.245885 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:15.245846    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db" event={"ID":"57dc7b45-6bde-44ce-ab43-7f38c5a19fae","Type":"ContainerStarted","Data":"a052fd8d98e5363c3617434cfa2d6f7750988f1caabb56e8a37d146513c59bd7"}
Jul 15 12:26:15.246348 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:15.245972    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:26:15.278140 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:15.278093    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db" podStartSLOduration=1.8414621169999998 podStartE2EDuration="5.278079356s" totalImagesPullingTime="3.436617239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:26:15.277823569 +0000 UTC m=+364.154546578" watchObservedRunningTime="2026-07-15 12:26:15.278079356 +0000 UTC m=+364.154802362"
Jul 15 12:26:21.227172 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:21.227137    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-595c7f4b4c-5nqsk"
Jul 15 12:26:46.252647 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:26:46.252613    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-679db"
Jul 15 12:27:34.379294 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.379215    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"]
Jul 15 12:27:34.382542 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.382517    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:34.385822 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.385797    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 15 12:27:34.385946 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.385836    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dft4g\""
Jul 15 12:27:34.386918 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.386900    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 15 12:27:34.387178 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.387163    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 15 12:27:34.393556 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.393518    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcllq\" (UniqueName: \"kubernetes.io/projected/66f97a63-1864-41a9-835f-9007bdb83527-kube-api-access-zcllq\") pod \"dns-operator-controller-manager-864dd78c4c-rhshn\" (UID: \"66f97a63-1864-41a9-835f-9007bdb83527\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:34.400032 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.400008    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"]
Jul 15 12:27:34.494437 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.494401    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcllq\" (UniqueName: \"kubernetes.io/projected/66f97a63-1864-41a9-835f-9007bdb83527-kube-api-access-zcllq\") pod \"dns-operator-controller-manager-864dd78c4c-rhshn\" (UID: \"66f97a63-1864-41a9-835f-9007bdb83527\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:34.518079 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.518038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcllq\" (UniqueName: \"kubernetes.io/projected/66f97a63-1864-41a9-835f-9007bdb83527-kube-api-access-zcllq\") pod \"dns-operator-controller-manager-864dd78c4c-rhshn\" (UID: \"66f97a63-1864-41a9-835f-9007bdb83527\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:34.692929 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.692893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:34.828209 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:34.828184    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"]
Jul 15 12:27:34.830904 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:27:34.830877    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f97a63_1864_41a9_835f_9007bdb83527.slice/crio-dfe192f1a22d3ce5df20747319a7bdedc84f9904b8be56024b00556e592d4bd9 WatchSource:0}: Error finding container dfe192f1a22d3ce5df20747319a7bdedc84f9904b8be56024b00556e592d4bd9: Status 404 returned error can't find the container with id dfe192f1a22d3ce5df20747319a7bdedc84f9904b8be56024b00556e592d4bd9
Jul 15 12:27:35.575223 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:35.575187    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn" event={"ID":"66f97a63-1864-41a9-835f-9007bdb83527","Type":"ContainerStarted","Data":"dfe192f1a22d3ce5df20747319a7bdedc84f9904b8be56024b00556e592d4bd9"}
Jul 15 12:27:37.585872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:37.585835    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn" event={"ID":"66f97a63-1864-41a9-835f-9007bdb83527","Type":"ContainerStarted","Data":"36e44b830b42927c100bf0dcb0bd04bfd3c28379808bb7d361dd873002af0d19"}
Jul 15 12:27:37.586250 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:37.585972    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:37.610011 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:37.609956    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn" podStartSLOduration=1.3654322159999999 podStartE2EDuration="3.609940426s" totalImagesPullingTime="2.24450821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:27:37.608387185 +0000 UTC m=+446.485110191" watchObservedRunningTime="2026-07-15 12:27:37.609940426 +0000 UTC m=+446.486663431"
Jul 15 12:27:39.451939 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.451908    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"]
Jul 15 12:27:39.455079 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.455060    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.460012 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.459993    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-htfqg\""
Jul 15 12:27:39.477225 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.477202    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"]
Jul 15 12:27:39.537346 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.537312    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnwjh\" (UniqueName: \"kubernetes.io/projected/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-kube-api-access-dnwjh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.537509 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.537361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.638441 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.638406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnwjh\" (UniqueName: \"kubernetes.io/projected/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-kube-api-access-dnwjh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.638644 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.638453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.638814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.638799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.653474 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.653448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnwjh\" (UniqueName: \"kubernetes.io/projected/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-kube-api-access-dnwjh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.765316 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.765236    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:39.911451 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:39.911426    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"]
Jul 15 12:27:39.913145 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:27:39.913115    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69be2ab2_bb0f_4dd9_8e6b_3f33a1818807.slice/crio-a66ddeac65c612dd15fcea71e2c10799949b4f5fddc2341028b30beab04f8dc0 WatchSource:0}: Error finding container a66ddeac65c612dd15fcea71e2c10799949b4f5fddc2341028b30beab04f8dc0: Status 404 returned error can't find the container with id a66ddeac65c612dd15fcea71e2c10799949b4f5fddc2341028b30beab04f8dc0
Jul 15 12:27:40.602508 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:40.602476    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk" event={"ID":"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807","Type":"ContainerStarted","Data":"a66ddeac65c612dd15fcea71e2c10799949b4f5fddc2341028b30beab04f8dc0"}
Jul 15 12:27:46.631774 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:46.631739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk" event={"ID":"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807","Type":"ContainerStarted","Data":"ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6"}
Jul 15 12:27:46.632131 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:46.631882    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:46.664083 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:46.664011    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk" podStartSLOduration=1.6498468929999999 podStartE2EDuration="7.663990469s" totalImagesPullingTime="6.014143576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:27:46.66275294 +0000 UTC m=+455.539475959" watchObservedRunningTime="2026-07-15 12:27:46.663990469 +0000 UTC m=+455.540713479"
Jul 15 12:27:48.593515 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:48.593485    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rhshn"
Jul 15 12:27:57.638879 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:57.638850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:59.680839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.680808    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"]
Jul 15 12:27:59.681261 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.680998    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" containerName="manager" containerID="cri-o://ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6" gracePeriod=2
Jul 15 12:27:59.694680 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:27:59.694639    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-67nlk\": pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:59.695477 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.695454    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"]
Jul 15 12:27:59.726732 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.726695    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"]
Jul 15 12:27:59.727230 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.727203    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" containerName="manager"
Jul 15 12:27:59.727230 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.727227    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" containerName="manager"
Jul 15 12:27:59.727389 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.727326    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" containerName="manager"
Jul 15 12:27:59.730316 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.730298    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.755077 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.755041    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"]
Jul 15 12:27:59.806144 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:27:59.806111    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:59.811512 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.811485    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tkdjn\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.811619 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.811524    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdktq\" (UniqueName: \"kubernetes.io/projected/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-kube-api-access-tdktq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tkdjn\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.912327 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.912293    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tkdjn\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.912466 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.912347    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdktq\" (UniqueName: \"kubernetes.io/projected/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-kube-api-access-tdktq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tkdjn\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.912702 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.912682    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tkdjn\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.927301 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.927267    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdktq\" (UniqueName: \"kubernetes.io/projected/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-kube-api-access-tdktq\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tkdjn\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:27:59.929085 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:27:59.929066    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:27:59.931200 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:27:59.931147    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:28:00.012858 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.012829    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-extensions-socket-volume\") pod \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") "
Jul 15 12:28:00.013023 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.012886    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dnwjh\" (UniqueName: \"kubernetes.io/projected/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-kube-api-access-dnwjh\") pod \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\" (UID: \"69be2ab2-bb0f-4dd9-8e6b-3f33a1818807\") "
Jul 15 12:28:00.013311 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.013286    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" (UID: "69be2ab2-bb0f-4dd9-8e6b-3f33a1818807"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:28:00.015097 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.015073    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-kube-api-access-dnwjh" (OuterVolumeSpecName: "kube-api-access-dnwjh") pod "69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" (UID: "69be2ab2-bb0f-4dd9-8e6b-3f33a1818807"). InnerVolumeSpecName "kube-api-access-dnwjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:28:00.074117 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.074079    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:28:00.114302 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.114271    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnwjh\" (UniqueName: \"kubernetes.io/projected/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-kube-api-access-dnwjh\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:28:00.114302 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.114298    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807-extensions-socket-volume\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:28:00.236113 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.236030    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"]
Jul 15 12:28:00.240172 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:28:00.240144    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d38dbc_6bca_4ba1_80f3_fd5225f0abc4.slice/crio-2ff90e3b9064eeee1790ea3d470275f5bfe247ad69b4f98a57f4c312aa1cdcd8 WatchSource:0}: Error finding container 2ff90e3b9064eeee1790ea3d470275f5bfe247ad69b4f98a57f4c312aa1cdcd8: Status 404 returned error can't find the container with id 2ff90e3b9064eeee1790ea3d470275f5bfe247ad69b4f98a57f4c312aa1cdcd8
Jul 15 12:28:00.700720 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.700685    2576 generic.go:356] "Generic (PLEG): container finished" podID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" containerID="ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6" exitCode=0
Jul 15 12:28:00.701146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.700757    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:28:00.701146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.700804    2576 scope.go:117] "RemoveContainer" containerID="ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6"
Jul 15 12:28:00.702974 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.702941    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn" event={"ID":"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4","Type":"ContainerStarted","Data":"a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654"}
Jul 15 12:28:00.703055 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.703001    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn" event={"ID":"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4","Type":"ContainerStarted","Data":"2ff90e3b9064eeee1790ea3d470275f5bfe247ad69b4f98a57f4c312aa1cdcd8"}
Jul 15 12:28:00.703104 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.703083    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:28:00.706427 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:00.706400    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:28:00.710611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.710595    2576 scope.go:117] "RemoveContainer" containerID="ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6"
Jul 15 12:28:00.710861 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:00.710845    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6\": container with ID starting with ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6 not found: ID does not exist" containerID="ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6"
Jul 15 12:28:00.710898 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.710871    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6"} err="failed to get container status \"ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6\": rpc error: code = NotFound desc = could not find container \"ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6\": container with ID starting with ce18021c6800f17c344620d6a3f181ea92031cb8d7ce534c99f0466a76bf5cc6 not found: ID does not exist"
Jul 15 12:28:00.765810 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:00.765751    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn" podStartSLOduration=1.7657293200000002 podStartE2EDuration="1.76572932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:00.763564229 +0000 UTC m=+469.640287235" watchObservedRunningTime="2026-07-15 12:28:00.76572932 +0000 UTC m=+469.642452327"
Jul 15 12:28:00.777806 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:00.777769    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:28:00.782613 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:00.782589    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:28:01.570794 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:01.570755    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67nlk\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67nlk"
Jul 15 12:28:01.571635 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:01.571615    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69be2ab2-bb0f-4dd9-8e6b-3f33a1818807" path="/var/lib/kubelet/pods/69be2ab2-bb0f-4dd9-8e6b-3f33a1818807/volumes"
Jul 15 12:28:11.711570 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:11.711506    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:28:29.854962 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:29.854927    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"]
Jul 15 12:28:29.855331 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:29.855123    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn" podUID="b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" containerName="manager" containerID="cri-o://a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654" gracePeriod=10
Jul 15 12:28:30.094092 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.094065    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:28:30.183741 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.183709    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-extensions-socket-volume\") pod \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") "
Jul 15 12:28:30.183916 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.183788    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdktq\" (UniqueName: \"kubernetes.io/projected/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-kube-api-access-tdktq\") pod \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\" (UID: \"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4\") "
Jul 15 12:28:30.184101 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.184066    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" (UID: "b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:28:30.185777 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.185755    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-kube-api-access-tdktq" (OuterVolumeSpecName: "kube-api-access-tdktq") pod "b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" (UID: "b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4"). InnerVolumeSpecName "kube-api-access-tdktq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:28:30.285213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.285177    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdktq\" (UniqueName: \"kubernetes.io/projected/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-kube-api-access-tdktq\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:28:30.285213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.285206    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4-extensions-socket-volume\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:28:30.833862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.833825    2576 generic.go:356] "Generic (PLEG): container finished" podID="b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" containerID="a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654" exitCode=0
Jul 15 12:28:30.834031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.833868    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"
Jul 15 12:28:30.834031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.833894    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn" event={"ID":"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4","Type":"ContainerDied","Data":"a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654"}
Jul 15 12:28:30.834031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.833943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn" event={"ID":"b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4","Type":"ContainerDied","Data":"2ff90e3b9064eeee1790ea3d470275f5bfe247ad69b4f98a57f4c312aa1cdcd8"}
Jul 15 12:28:30.834031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.833965    2576 scope.go:117] "RemoveContainer" containerID="a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654"
Jul 15 12:28:30.842790 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.842774    2576 scope.go:117] "RemoveContainer" containerID="a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654"
Jul 15 12:28:30.843040 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:30.843013    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654\": container with ID starting with a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654 not found: ID does not exist" containerID="a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654"
Jul 15 12:28:30.843167 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.843038    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654"} err="failed to get container status \"a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654\": rpc error: code = NotFound desc = could not find container \"a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654\": container with ID starting with a381d3e440d7b9cf00a70a66f101ce7960e37be499e6aaf876443b83f6053654 not found: ID does not exist"
Jul 15 12:28:30.858225 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.858198    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"]
Jul 15 12:28:30.871301 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:30.871280    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tkdjn"]
Jul 15 12:28:31.571798 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:31.571762    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" path="/var/lib/kubelet/pods/b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4/volumes"
Jul 15 12:28:46.308787 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.308756    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"]
Jul 15 12:28:46.309256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.309055    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" containerName="manager"
Jul 15 12:28:46.309256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.309066    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" containerName="manager"
Jul 15 12:28:46.309256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.309130    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2d38dbc-6bca-4ba1-80f3-fd5225f0abc4" containerName="manager"
Jul 15 12:28:46.312172 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.312153    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.317144 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.317125    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 15 12:28:46.318322 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.318302    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4v84j\""
Jul 15 12:28:46.334139 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.334111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"]
Jul 15 12:28:46.426400 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426400 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426410    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426642 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426440    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426642 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426503    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426642 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426600    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dbl4\" (UniqueName: \"kubernetes.io/projected/f6e88f17-42ee-47aa-b377-572cee4dcb0f-kube-api-access-9dbl4\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426642 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426630    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426766 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426672    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426766 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.426829 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.426771    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.527737 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.527701    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.527737 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.527741    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.527936 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.527758    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.527984 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.527959    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dbl4\" (UniqueName: \"kubernetes.io/projected/f6e88f17-42ee-47aa-b377-572cee4dcb0f-kube-api-access-9dbl4\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528030 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.527994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528079 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528079 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528056    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528164 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528108    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528164 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528257 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528257 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528205    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528428 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528404    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.528617 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.528486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.530377 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.530354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.530628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.530610    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.538704 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.538679    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dbl4\" (UniqueName: \"kubernetes.io/projected/f6e88f17-42ee-47aa-b377-572cee4dcb0f-kube-api-access-9dbl4\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.546955 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.546932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e88f17-42ee-47aa-b377-572cee4dcb0f-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-kfflk\" (UID: \"f6e88f17-42ee-47aa-b377-572cee4dcb0f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.624447 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.624361    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:46.775479 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.775452    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"]
Jul 15 12:28:46.778100 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:28:46.778067    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e88f17_42ee_47aa_b377_572cee4dcb0f.slice/crio-9bcc2a93a8aff7d3955bfa575d96f8e99c905e91d148060029980174c818dd26 WatchSource:0}: Error finding container 9bcc2a93a8aff7d3955bfa575d96f8e99c905e91d148060029980174c818dd26: Status 404 returned error can't find the container with id 9bcc2a93a8aff7d3955bfa575d96f8e99c905e91d148060029980174c818dd26
Jul 15 12:28:46.903181 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:46.903144    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" event={"ID":"f6e88f17-42ee-47aa-b377-572cee4dcb0f","Type":"ContainerStarted","Data":"9bcc2a93a8aff7d3955bfa575d96f8e99c905e91d148060029980174c818dd26"}
Jul 15 12:28:49.395343 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:49.395303    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:28:49.395625 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:49.395387    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:28:49.395625 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:49.395416    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:28:49.925238 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:49.925197    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" event={"ID":"f6e88f17-42ee-47aa-b377-572cee4dcb0f","Type":"ContainerStarted","Data":"7420675cea1b40785a7a244ca03551855899281d801c8f44ff22f311c9391e7d"}
Jul 15 12:28:49.952173 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:49.952122    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" podStartSLOduration=1.337089327 podStartE2EDuration="3.952105685s" totalImagesPullingTime="2.615016358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:49.950364048 +0000 UTC m=+518.827087053" watchObservedRunningTime="2026-07-15 12:28:49.952105685 +0000 UTC m=+518.828828691"
Jul 15 12:28:50.625254 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:50.625206    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:50.630418 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:50.630395    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:50.930351 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:50.930320    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:50.931271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:50.931253    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:28:51.051696 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.051663    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:28:51.055281 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.055265    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.058030 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.058010    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 15 12:28:51.058502 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.058483    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9hp94\""
Jul 15 12:28:51.073808 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.073784    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:28:51.144349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.144318    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:28:51.170043 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.170010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjwgq\" (UniqueName: \"kubernetes.io/projected/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-kube-api-access-cjwgq\") pod \"limitador-limitador-64c94fd64f-x6rfl\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.170186 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.170115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-config-file\") pod \"limitador-limitador-64c94fd64f-x6rfl\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.271394 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.271297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-config-file\") pod \"limitador-limitador-64c94fd64f-x6rfl\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.271394 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.271375    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjwgq\" (UniqueName: \"kubernetes.io/projected/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-kube-api-access-cjwgq\") pod \"limitador-limitador-64c94fd64f-x6rfl\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.271924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.271904    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-config-file\") pod \"limitador-limitador-64c94fd64f-x6rfl\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.279969 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.279943    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjwgq\" (UniqueName: \"kubernetes.io/projected/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-kube-api-access-cjwgq\") pod \"limitador-limitador-64c94fd64f-x6rfl\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.365981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.365938    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:51.502704 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.502680    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:28:51.504699 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:28:51.504662    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4231c627_ea2c_4d7d_aa5f_b1f3fd10f5e1.slice/crio-6bca9e4703e617be07b25ab251d4998dc43652d722d89b64ef3bd4c9227fdc60 WatchSource:0}: Error finding container 6bca9e4703e617be07b25ab251d4998dc43652d722d89b64ef3bd4c9227fdc60: Status 404 returned error can't find the container with id 6bca9e4703e617be07b25ab251d4998dc43652d722d89b64ef3bd4c9227fdc60
Jul 15 12:28:51.890035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.889999    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hb76w"]
Jul 15 12:28:51.894616 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.894598    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:51.897289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.897269    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ffgw8\""
Jul 15 12:28:51.904340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.904294    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hb76w"]
Jul 15 12:28:51.936245 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.936205    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl" event={"ID":"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1","Type":"ContainerStarted","Data":"6bca9e4703e617be07b25ab251d4998dc43652d722d89b64ef3bd4c9227fdc60"}
Jul 15 12:28:51.977020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:51.976986    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf4bt\" (UniqueName: \"kubernetes.io/projected/5b54020b-0df4-48a6-ad09-5e1ea2598142-kube-api-access-wf4bt\") pod \"authorino-58cd955675-hb76w\" (UID: \"5b54020b-0df4-48a6-ad09-5e1ea2598142\") " pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:52.078149 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:52.078116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf4bt\" (UniqueName: \"kubernetes.io/projected/5b54020b-0df4-48a6-ad09-5e1ea2598142-kube-api-access-wf4bt\") pod \"authorino-58cd955675-hb76w\" (UID: \"5b54020b-0df4-48a6-ad09-5e1ea2598142\") " pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:52.090775 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:52.090743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf4bt\" (UniqueName: \"kubernetes.io/projected/5b54020b-0df4-48a6-ad09-5e1ea2598142-kube-api-access-wf4bt\") pod \"authorino-58cd955675-hb76w\" (UID: \"5b54020b-0df4-48a6-ad09-5e1ea2598142\") " pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:52.205330 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:52.205291    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:52.381184 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:52.380238    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hb76w"]
Jul 15 12:28:52.942599 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:52.942567    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hb76w" event={"ID":"5b54020b-0df4-48a6-ad09-5e1ea2598142","Type":"ContainerStarted","Data":"cb38a638bdb5d8d74a3bd2066cc6f9035f49b6fdecc08e9e09a07f5ae4b9577b"}
Jul 15 12:28:56.787569 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:56.787522    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hb76w"]
Jul 15 12:28:56.967207 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:56.967162    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hb76w" event={"ID":"5b54020b-0df4-48a6-ad09-5e1ea2598142","Type":"ContainerStarted","Data":"5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14"}
Jul 15 12:28:56.968520 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:56.968497    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl" event={"ID":"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1","Type":"ContainerStarted","Data":"ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f"}
Jul 15 12:28:56.968671 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:56.968573    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:28:56.996154 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:56.996103    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hb76w" podStartSLOduration=2.338249281 podStartE2EDuration="5.996088781s" totalImagesPullingTime="3.6578395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:56.995829836 +0000 UTC m=+525.872552841" watchObservedRunningTime="2026-07-15 12:28:56.996088781 +0000 UTC m=+525.872811806"
Jul 15 12:28:57.043315 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:57.043212    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl" podStartSLOduration=1.562347338 podStartE2EDuration="6.04319458s" totalImagesPullingTime="4.480847242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:28:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:28:57.041922068 +0000 UTC m=+525.918645074" watchObservedRunningTime="2026-07-15 12:28:57.04319458 +0000 UTC m=+525.919917587"
Jul 15 12:28:57.973185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:57.973144    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hb76w" podUID="5b54020b-0df4-48a6-ad09-5e1ea2598142" containerName="authorino" containerID="cri-o://5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14" gracePeriod=30
Jul 15 12:28:58.228250 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.228197    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:58.332920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.332881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf4bt\" (UniqueName: \"kubernetes.io/projected/5b54020b-0df4-48a6-ad09-5e1ea2598142-kube-api-access-wf4bt\") pod \"5b54020b-0df4-48a6-ad09-5e1ea2598142\" (UID: \"5b54020b-0df4-48a6-ad09-5e1ea2598142\") "
Jul 15 12:28:58.334931 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.334908    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b54020b-0df4-48a6-ad09-5e1ea2598142-kube-api-access-wf4bt" (OuterVolumeSpecName: "kube-api-access-wf4bt") pod "5b54020b-0df4-48a6-ad09-5e1ea2598142" (UID: "5b54020b-0df4-48a6-ad09-5e1ea2598142"). InnerVolumeSpecName "kube-api-access-wf4bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:28:58.434280 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.434248    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf4bt\" (UniqueName: \"kubernetes.io/projected/5b54020b-0df4-48a6-ad09-5e1ea2598142-kube-api-access-wf4bt\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:28:58.978635 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.978599    2576 generic.go:356] "Generic (PLEG): container finished" podID="5b54020b-0df4-48a6-ad09-5e1ea2598142" containerID="5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14" exitCode=0
Jul 15 12:28:58.979059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.978641    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hb76w"
Jul 15 12:28:58.979059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.978689    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hb76w" event={"ID":"5b54020b-0df4-48a6-ad09-5e1ea2598142","Type":"ContainerDied","Data":"5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14"}
Jul 15 12:28:58.979059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.978727    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hb76w" event={"ID":"5b54020b-0df4-48a6-ad09-5e1ea2598142","Type":"ContainerDied","Data":"cb38a638bdb5d8d74a3bd2066cc6f9035f49b6fdecc08e9e09a07f5ae4b9577b"}
Jul 15 12:28:58.979059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.978742    2576 scope.go:117] "RemoveContainer" containerID="5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14"
Jul 15 12:28:58.988145 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.987943    2576 scope.go:117] "RemoveContainer" containerID="5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14"
Jul 15 12:28:58.988805 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:28:58.988582    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14\": container with ID starting with 5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14 not found: ID does not exist" containerID="5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14"
Jul 15 12:28:58.988805 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:58.988626    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14"} err="failed to get container status \"5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14\": rpc error: code = NotFound desc = could not find container \"5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14\": container with ID starting with 5de56c2e25bee228a5f0b65631ee8d4aa1783f9cc2cd655a68578a80e7143b14 not found: ID does not exist"
Jul 15 12:28:59.010856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:59.010830    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hb76w"]
Jul 15 12:28:59.019331 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:59.019302    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hb76w"]
Jul 15 12:28:59.572029 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:28:59.571993    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b54020b-0df4-48a6-ad09-5e1ea2598142" path="/var/lib/kubelet/pods/5b54020b-0df4-48a6-ad09-5e1ea2598142/volumes"
Jul 15 12:29:07.837671 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:07.837589    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:29:07.838078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:07.837772    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl" podUID="4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" containerName="limitador" containerID="cri-o://ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f" gracePeriod=30
Jul 15 12:29:07.838429 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:07.838408    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:29:08.378835 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.378811    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:29:08.419117 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.419096    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-config-file\") pod \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") "
Jul 15 12:29:08.419230 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.419152    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjwgq\" (UniqueName: \"kubernetes.io/projected/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-kube-api-access-cjwgq\") pod \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\" (UID: \"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1\") "
Jul 15 12:29:08.419422 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.419399    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-config-file" (OuterVolumeSpecName: "config-file") pod "4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" (UID: "4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:29:08.421068 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.421045    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-kube-api-access-cjwgq" (OuterVolumeSpecName: "kube-api-access-cjwgq") pod "4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" (UID: "4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1"). InnerVolumeSpecName "kube-api-access-cjwgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:29:08.519613 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.519584    2576 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-config-file\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:29:08.519613 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:08.519611    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjwgq\" (UniqueName: \"kubernetes.io/projected/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1-kube-api-access-cjwgq\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:29:09.023818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.023785    2576 generic.go:356] "Generic (PLEG): container finished" podID="4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" containerID="ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f" exitCode=0
Jul 15 12:29:09.024190 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.023826    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"
Jul 15 12:29:09.024190 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.023860    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl" event={"ID":"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1","Type":"ContainerDied","Data":"ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f"}
Jul 15 12:29:09.024190 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.023898    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6rfl" event={"ID":"4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1","Type":"ContainerDied","Data":"6bca9e4703e617be07b25ab251d4998dc43652d722d89b64ef3bd4c9227fdc60"}
Jul 15 12:29:09.024190 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.023919    2576 scope.go:117] "RemoveContainer" containerID="ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f"
Jul 15 12:29:09.032402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.032384    2576 scope.go:117] "RemoveContainer" containerID="ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f"
Jul 15 12:29:09.032663 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:29:09.032645    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f\": container with ID starting with ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f not found: ID does not exist" containerID="ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f"
Jul 15 12:29:09.032729 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.032676    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f"} err="failed to get container status \"ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f\": rpc error: code = NotFound desc = could not find container \"ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f\": container with ID starting with ad5a8b867cc474138f8d2197c62d877a003e37945eaf037256df275b208b2d2f not found: ID does not exist"
Jul 15 12:29:09.055089 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.055062    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:29:09.059134 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.059112    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6rfl"]
Jul 15 12:29:09.572393 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:09.572361    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" path="/var/lib/kubelet/pods/4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1/volumes"
Jul 15 12:29:12.974169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974134    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"]
Jul 15 12:29:12.974621 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974610    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" containerName="limitador"
Jul 15 12:29:12.974694 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974628    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" containerName="limitador"
Jul 15 12:29:12.974694 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974659    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b54020b-0df4-48a6-ad09-5e1ea2598142" containerName="authorino"
Jul 15 12:29:12.974694 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974667    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b54020b-0df4-48a6-ad09-5e1ea2598142" containerName="authorino"
Jul 15 12:29:12.974838 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974755    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b54020b-0df4-48a6-ad09-5e1ea2598142" containerName="authorino"
Jul 15 12:29:12.974838 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.974766    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4231c627-ea2c-4d7d-aa5f-b1f3fd10f5e1" containerName="limitador"
Jul 15 12:29:12.978831 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.978812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:12.983611 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.983592    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-jkrpf\""
Jul 15 12:29:12.984521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.984503    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 15 12:29:12.985103 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.985084    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 15 12:29:12.985705 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:12.985675    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 15 12:29:13.018767 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.018748    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"]
Jul 15 12:29:13.047504 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.047482    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/894eecde-850c-4947-b9be-95d7a61051fa-data\") pod \"postgres-69fb4594cf-f5kf7\" (UID: \"894eecde-850c-4947-b9be-95d7a61051fa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.047644 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.047516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc59p\" (UniqueName: \"kubernetes.io/projected/894eecde-850c-4947-b9be-95d7a61051fa-kube-api-access-rc59p\") pod \"postgres-69fb4594cf-f5kf7\" (UID: \"894eecde-850c-4947-b9be-95d7a61051fa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.148816 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.148790    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc59p\" (UniqueName: \"kubernetes.io/projected/894eecde-850c-4947-b9be-95d7a61051fa-kube-api-access-rc59p\") pod \"postgres-69fb4594cf-f5kf7\" (UID: \"894eecde-850c-4947-b9be-95d7a61051fa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.148939 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.148874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/894eecde-850c-4947-b9be-95d7a61051fa-data\") pod \"postgres-69fb4594cf-f5kf7\" (UID: \"894eecde-850c-4947-b9be-95d7a61051fa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.149168 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.149152    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/894eecde-850c-4947-b9be-95d7a61051fa-data\") pod \"postgres-69fb4594cf-f5kf7\" (UID: \"894eecde-850c-4947-b9be-95d7a61051fa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.167411 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.167387    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc59p\" (UniqueName: \"kubernetes.io/projected/894eecde-850c-4947-b9be-95d7a61051fa-kube-api-access-rc59p\") pod \"postgres-69fb4594cf-f5kf7\" (UID: \"894eecde-850c-4947-b9be-95d7a61051fa\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.288779 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.288722    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:13.418793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:13.418766    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"]
Jul 15 12:29:13.421037 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:29:13.421012    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894eecde_850c_4947_b9be_95d7a61051fa.slice/crio-34299d789bb530a6a150cd74dab3d53561f5e07112df3b95027d5a56f69b9a4b WatchSource:0}: Error finding container 34299d789bb530a6a150cd74dab3d53561f5e07112df3b95027d5a56f69b9a4b: Status 404 returned error can't find the container with id 34299d789bb530a6a150cd74dab3d53561f5e07112df3b95027d5a56f69b9a4b
Jul 15 12:29:14.046852 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:14.046819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7" event={"ID":"894eecde-850c-4947-b9be-95d7a61051fa","Type":"ContainerStarted","Data":"34299d789bb530a6a150cd74dab3d53561f5e07112df3b95027d5a56f69b9a4b"}
Jul 15 12:29:21.083983 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:21.083951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7" event={"ID":"894eecde-850c-4947-b9be-95d7a61051fa","Type":"ContainerStarted","Data":"d85da34b0961c870a8c474c63394c9ff515041dee113634b882cb1e08f407dd7"}
Jul 15 12:29:21.084462 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:21.084034    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:21.104363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:21.104320    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7" podStartSLOduration=1.619185433 podStartE2EDuration="9.104306288s" totalImagesPullingTime="7.485120855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:21.103329127 +0000 UTC m=+549.980052143" watchObservedRunningTime="2026-07-15 12:29:21.104306288 +0000 UTC m=+549.981029294"
Jul 15 12:29:27.117976 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.117941    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-f5kf7"
Jul 15 12:29:27.705469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.705433    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-29ql9"]
Jul 15 12:29:27.709006 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.708985    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:27.711659 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.711624    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ffgw8\""
Jul 15 12:29:27.717009 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.716984    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-29ql9"]
Jul 15 12:29:27.767277 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.767236    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dnfd\" (UniqueName: \"kubernetes.io/projected/c9f83994-5db2-4bd2-82d5-db3857e37bc5-kube-api-access-4dnfd\") pod \"authorino-5d4b67c446-29ql9\" (UID: \"c9f83994-5db2-4bd2-82d5-db3857e37bc5\") " pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:27.867711 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.867665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dnfd\" (UniqueName: \"kubernetes.io/projected/c9f83994-5db2-4bd2-82d5-db3857e37bc5-kube-api-access-4dnfd\") pod \"authorino-5d4b67c446-29ql9\" (UID: \"c9f83994-5db2-4bd2-82d5-db3857e37bc5\") " pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:27.878481 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.878449    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dnfd\" (UniqueName: \"kubernetes.io/projected/c9f83994-5db2-4bd2-82d5-db3857e37bc5-kube-api-access-4dnfd\") pod \"authorino-5d4b67c446-29ql9\" (UID: \"c9f83994-5db2-4bd2-82d5-db3857e37bc5\") " pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:27.956618 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.956485    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-29ql9"]
Jul 15 12:29:27.956764 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:27.956746    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:28.001626 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.001592    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b466ddb45-q9n88"]
Jul 15 12:29:28.006390 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.006021    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:28.015061 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.015002    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b466ddb45-q9n88"]
Jul 15 12:29:28.070314 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.070278    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qq6\" (UniqueName: \"kubernetes.io/projected/077808ff-4fa1-42bc-84c2-d4a7fdd0d857-kube-api-access-n7qq6\") pod \"authorino-b466ddb45-q9n88\" (UID: \"077808ff-4fa1-42bc-84c2-d4a7fdd0d857\") " pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:28.096222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.096188    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-29ql9"]
Jul 15 12:29:28.099179 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:29:28.099143    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9f83994_5db2_4bd2_82d5_db3857e37bc5.slice/crio-cf9027aa71032e6d79ca167466c431b98380c554c1980fb39c5c16eadcc52c16 WatchSource:0}: Error finding container cf9027aa71032e6d79ca167466c431b98380c554c1980fb39c5c16eadcc52c16: Status 404 returned error can't find the container with id cf9027aa71032e6d79ca167466c431b98380c554c1980fb39c5c16eadcc52c16
Jul 15 12:29:28.113835 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.113809    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-29ql9" event={"ID":"c9f83994-5db2-4bd2-82d5-db3857e37bc5","Type":"ContainerStarted","Data":"cf9027aa71032e6d79ca167466c431b98380c554c1980fb39c5c16eadcc52c16"}
Jul 15 12:29:28.171407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.171365    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qq6\" (UniqueName: \"kubernetes.io/projected/077808ff-4fa1-42bc-84c2-d4a7fdd0d857-kube-api-access-n7qq6\") pod \"authorino-b466ddb45-q9n88\" (UID: \"077808ff-4fa1-42bc-84c2-d4a7fdd0d857\") " pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:28.182333 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.182309    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qq6\" (UniqueName: \"kubernetes.io/projected/077808ff-4fa1-42bc-84c2-d4a7fdd0d857-kube-api-access-n7qq6\") pod \"authorino-b466ddb45-q9n88\" (UID: \"077808ff-4fa1-42bc-84c2-d4a7fdd0d857\") " pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:28.201633 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.201600    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b466ddb45-q9n88"]
Jul 15 12:29:28.201868 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.201855    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:28.335372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:28.335349    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b466ddb45-q9n88"]
Jul 15 12:29:28.337330 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:29:28.337291    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod077808ff_4fa1_42bc_84c2_d4a7fdd0d857.slice/crio-d1dc7c2950962ebfc8f04c3cee5e770bcdea7c6d2dc2e01c673c016da074db70 WatchSource:0}: Error finding container d1dc7c2950962ebfc8f04c3cee5e770bcdea7c6d2dc2e01c673c016da074db70: Status 404 returned error can't find the container with id d1dc7c2950962ebfc8f04c3cee5e770bcdea7c6d2dc2e01c673c016da074db70
Jul 15 12:29:29.119891 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.119850    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b466ddb45-q9n88" event={"ID":"077808ff-4fa1-42bc-84c2-d4a7fdd0d857","Type":"ContainerStarted","Data":"d1dc7c2950962ebfc8f04c3cee5e770bcdea7c6d2dc2e01c673c016da074db70"}
Jul 15 12:29:29.121212 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.121191    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-29ql9" event={"ID":"c9f83994-5db2-4bd2-82d5-db3857e37bc5","Type":"ContainerStarted","Data":"6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea"}
Jul 15 12:29:29.121327 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.121259    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-29ql9" podUID="c9f83994-5db2-4bd2-82d5-db3857e37bc5" containerName="authorino" containerID="cri-o://6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea" gracePeriod=30
Jul 15 12:29:29.138347 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.138293    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-29ql9" podStartSLOduration=1.738996072 podStartE2EDuration="2.13827744s" totalImagesPullingTime="399.281368ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:29.136920924 +0000 UTC m=+558.013643930" watchObservedRunningTime="2026-07-15 12:29:29.13827744 +0000 UTC m=+558.015000446"
Jul 15 12:29:29.380078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.380015    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:29.482918 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.482883    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dnfd\" (UniqueName: \"kubernetes.io/projected/c9f83994-5db2-4bd2-82d5-db3857e37bc5-kube-api-access-4dnfd\") pod \"c9f83994-5db2-4bd2-82d5-db3857e37bc5\" (UID: \"c9f83994-5db2-4bd2-82d5-db3857e37bc5\") "
Jul 15 12:29:29.484968 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.484939    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f83994-5db2-4bd2-82d5-db3857e37bc5-kube-api-access-4dnfd" (OuterVolumeSpecName: "kube-api-access-4dnfd") pod "c9f83994-5db2-4bd2-82d5-db3857e37bc5" (UID: "c9f83994-5db2-4bd2-82d5-db3857e37bc5"). InnerVolumeSpecName "kube-api-access-4dnfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:29:29.583475 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:29.583443    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dnfd\" (UniqueName: \"kubernetes.io/projected/c9f83994-5db2-4bd2-82d5-db3857e37bc5-kube-api-access-4dnfd\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:29:30.126953 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.126914    2576 generic.go:356] "Generic (PLEG): container finished" podID="c9f83994-5db2-4bd2-82d5-db3857e37bc5" containerID="6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea" exitCode=0
Jul 15 12:29:30.127146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.126980    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-29ql9"
Jul 15 12:29:30.127146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.126996    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-29ql9" event={"ID":"c9f83994-5db2-4bd2-82d5-db3857e37bc5","Type":"ContainerDied","Data":"6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea"}
Jul 15 12:29:30.127146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.127037    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-29ql9" event={"ID":"c9f83994-5db2-4bd2-82d5-db3857e37bc5","Type":"ContainerDied","Data":"cf9027aa71032e6d79ca167466c431b98380c554c1980fb39c5c16eadcc52c16"}
Jul 15 12:29:30.127146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.127054    2576 scope.go:117] "RemoveContainer" containerID="6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea"
Jul 15 12:29:30.135301 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.135274    2576 scope.go:117] "RemoveContainer" containerID="6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea"
Jul 15 12:29:30.135582 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:29:30.135564    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea\": container with ID starting with 6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea not found: ID does not exist" containerID="6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea"
Jul 15 12:29:30.135657 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.135595    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea"} err="failed to get container status \"6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea\": rpc error: code = NotFound desc = could not find container \"6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea\": container with ID starting with 6b9a9cbc467d302d652f314c161b70605e22ab0857fd19117f554d57f69844ea not found: ID does not exist"
Jul 15 12:29:30.143235 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.143212    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-29ql9"]
Jul 15 12:29:30.148385 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:30.148360    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-29ql9"]
Jul 15 12:29:31.574362 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:31.574318    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f83994-5db2-4bd2-82d5-db3857e37bc5" path="/var/lib/kubelet/pods/c9f83994-5db2-4bd2-82d5-db3857e37bc5/volumes"
Jul 15 12:29:32.139312 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:32.139279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b466ddb45-q9n88" event={"ID":"077808ff-4fa1-42bc-84c2-d4a7fdd0d857","Type":"ContainerStarted","Data":"290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11"}
Jul 15 12:29:32.139312 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:32.139300    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b466ddb45-q9n88" podUID="077808ff-4fa1-42bc-84c2-d4a7fdd0d857" containerName="authorino" containerID="cri-o://290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11" gracePeriod=30
Jul 15 12:29:32.391315 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:32.391260    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:32.506668 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:32.506630    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7qq6\" (UniqueName: \"kubernetes.io/projected/077808ff-4fa1-42bc-84c2-d4a7fdd0d857-kube-api-access-n7qq6\") pod \"077808ff-4fa1-42bc-84c2-d4a7fdd0d857\" (UID: \"077808ff-4fa1-42bc-84c2-d4a7fdd0d857\") "
Jul 15 12:29:32.508711 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:32.508687    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/077808ff-4fa1-42bc-84c2-d4a7fdd0d857-kube-api-access-n7qq6" (OuterVolumeSpecName: "kube-api-access-n7qq6") pod "077808ff-4fa1-42bc-84c2-d4a7fdd0d857" (UID: "077808ff-4fa1-42bc-84c2-d4a7fdd0d857"). InnerVolumeSpecName "kube-api-access-n7qq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:29:32.608164 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:32.608128    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7qq6\" (UniqueName: \"kubernetes.io/projected/077808ff-4fa1-42bc-84c2-d4a7fdd0d857-kube-api-access-n7qq6\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:29:33.144946 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.144913    2576 generic.go:356] "Generic (PLEG): container finished" podID="077808ff-4fa1-42bc-84c2-d4a7fdd0d857" containerID="290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11" exitCode=0
Jul 15 12:29:33.145148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.144958    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b466ddb45-q9n88"
Jul 15 12:29:33.145148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.144978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b466ddb45-q9n88" event={"ID":"077808ff-4fa1-42bc-84c2-d4a7fdd0d857","Type":"ContainerDied","Data":"290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11"}
Jul 15 12:29:33.145148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.145011    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b466ddb45-q9n88" event={"ID":"077808ff-4fa1-42bc-84c2-d4a7fdd0d857","Type":"ContainerDied","Data":"d1dc7c2950962ebfc8f04c3cee5e770bcdea7c6d2dc2e01c673c016da074db70"}
Jul 15 12:29:33.145148 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.145032    2576 scope.go:117] "RemoveContainer" containerID="290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11"
Jul 15 12:29:33.153237 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.153219    2576 scope.go:117] "RemoveContainer" containerID="290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11"
Jul 15 12:29:33.153476 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:29:33.153458    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11\": container with ID starting with 290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11 not found: ID does not exist" containerID="290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11"
Jul 15 12:29:33.153525 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.153485    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11"} err="failed to get container status \"290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11\": rpc error: code = NotFound desc = could not find container \"290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11\": container with ID starting with 290d4725637319ca3855ff38fee4c07313b44d65f9021afbb91f09e915f92c11 not found: ID does not exist"
Jul 15 12:29:33.168659 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.168631    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b466ddb45-q9n88"]
Jul 15 12:29:33.172724 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.172704    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b466ddb45-q9n88"]
Jul 15 12:29:33.576900 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:33.571969    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="077808ff-4fa1-42bc-84c2-d4a7fdd0d857" path="/var/lib/kubelet/pods/077808ff-4fa1-42bc-84c2-d4a7fdd0d857/volumes"
Jul 15 12:29:34.995245 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995216    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c4f9ff7b7-7h987"]
Jul 15 12:29:34.995595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995558    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f83994-5db2-4bd2-82d5-db3857e37bc5" containerName="authorino"
Jul 15 12:29:34.995595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995570    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f83994-5db2-4bd2-82d5-db3857e37bc5" containerName="authorino"
Jul 15 12:29:34.995595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995590    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="077808ff-4fa1-42bc-84c2-d4a7fdd0d857" containerName="authorino"
Jul 15 12:29:34.995595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995595    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="077808ff-4fa1-42bc-84c2-d4a7fdd0d857" containerName="authorino"
Jul 15 12:29:34.995740 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995646    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="077808ff-4fa1-42bc-84c2-d4a7fdd0d857" containerName="authorino"
Jul 15 12:29:34.995740 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.995658    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f83994-5db2-4bd2-82d5-db3857e37bc5" containerName="authorino"
Jul 15 12:29:34.999785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:34.999762    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.004169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.004149    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4qlgc\""
Jul 15 12:29:35.004306 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.004168    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 12:29:35.004306 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.004293    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 12:29:35.013954 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.013928    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c4f9ff7b7-7h987"]
Jul 15 12:29:35.127948 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.127911    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9defcc0a-a8e3-4722-8da0-a55911b4ac47-webhook-cert\") pod \"maas-controller-5c4f9ff7b7-7h987\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") " pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.128134 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.127963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pr6r\" (UniqueName: \"kubernetes.io/projected/9defcc0a-a8e3-4722-8da0-a55911b4ac47-kube-api-access-2pr6r\") pod \"maas-controller-5c4f9ff7b7-7h987\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") " pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.206568 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.206522    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-h9bb7"]
Jul 15 12:29:35.209820 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.209805    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.220489 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.220461    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-h9bb7"]
Jul 15 12:29:35.228592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.228569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pr6r\" (UniqueName: \"kubernetes.io/projected/9defcc0a-a8e3-4722-8da0-a55911b4ac47-kube-api-access-2pr6r\") pod \"maas-controller-5c4f9ff7b7-7h987\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") " pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.228721 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.228679    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9defcc0a-a8e3-4722-8da0-a55911b4ac47-webhook-cert\") pod \"maas-controller-5c4f9ff7b7-7h987\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") " pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.231080 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.231049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9defcc0a-a8e3-4722-8da0-a55911b4ac47-webhook-cert\") pod \"maas-controller-5c4f9ff7b7-7h987\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") " pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.239039 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.239018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pr6r\" (UniqueName: \"kubernetes.io/projected/9defcc0a-a8e3-4722-8da0-a55911b4ac47-kube-api-access-2pr6r\") pod \"maas-controller-5c4f9ff7b7-7h987\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") " pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.312121 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.312027    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:35.330012 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.329984    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c8e0a3-d7ff-4f61-853b-39c06407b827-webhook-cert\") pod \"maas-controller-74fcfc6ff5-h9bb7\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") " pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.330163 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.330032    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdbcp\" (UniqueName: \"kubernetes.io/projected/a0c8e0a3-d7ff-4f61-853b-39c06407b827-kube-api-access-wdbcp\") pod \"maas-controller-74fcfc6ff5-h9bb7\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") " pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.430801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.430773    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c8e0a3-d7ff-4f61-853b-39c06407b827-webhook-cert\") pod \"maas-controller-74fcfc6ff5-h9bb7\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") " pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.430946 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.430813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdbcp\" (UniqueName: \"kubernetes.io/projected/a0c8e0a3-d7ff-4f61-853b-39c06407b827-kube-api-access-wdbcp\") pod \"maas-controller-74fcfc6ff5-h9bb7\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") " pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.433102 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.433080    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c8e0a3-d7ff-4f61-853b-39c06407b827-webhook-cert\") pod \"maas-controller-74fcfc6ff5-h9bb7\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") " pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.440310 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.440281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdbcp\" (UniqueName: \"kubernetes.io/projected/a0c8e0a3-d7ff-4f61-853b-39c06407b827-kube-api-access-wdbcp\") pod \"maas-controller-74fcfc6ff5-h9bb7\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") " pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.446145 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.446125    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c4f9ff7b7-7h987"]
Jul 15 12:29:35.447912 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:29:35.447887    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9defcc0a_a8e3_4722_8da0_a55911b4ac47.slice/crio-e0f64301afe78c9ee49d003799c99125e609fc532623d69872d806c3de7e77fe WatchSource:0}: Error finding container e0f64301afe78c9ee49d003799c99125e609fc532623d69872d806c3de7e77fe: Status 404 returned error can't find the container with id e0f64301afe78c9ee49d003799c99125e609fc532623d69872d806c3de7e77fe
Jul 15 12:29:35.519589 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.519557    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:35.650617 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:35.650593    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-h9bb7"]
Jul 15 12:29:35.652706 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:29:35.652671    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c8e0a3_d7ff_4f61_853b_39c06407b827.slice/crio-5569b1307cfd5d986fdec99796f28180a3ac1724e7b788d702f5a051060abca9 WatchSource:0}: Error finding container 5569b1307cfd5d986fdec99796f28180a3ac1724e7b788d702f5a051060abca9: Status 404 returned error can't find the container with id 5569b1307cfd5d986fdec99796f28180a3ac1724e7b788d702f5a051060abca9
Jul 15 12:29:36.166457 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:36.166273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7" event={"ID":"a0c8e0a3-d7ff-4f61-853b-39c06407b827","Type":"ContainerStarted","Data":"5569b1307cfd5d986fdec99796f28180a3ac1724e7b788d702f5a051060abca9"}
Jul 15 12:29:36.168135 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:36.168108    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987" event={"ID":"9defcc0a-a8e3-4722-8da0-a55911b4ac47","Type":"ContainerStarted","Data":"e0f64301afe78c9ee49d003799c99125e609fc532623d69872d806c3de7e77fe"}
Jul 15 12:29:39.186458 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:39.186424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7" event={"ID":"a0c8e0a3-d7ff-4f61-853b-39c06407b827","Type":"ContainerStarted","Data":"0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d"}
Jul 15 12:29:39.186893 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:39.186571    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:39.187920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:39.187899    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987" event={"ID":"9defcc0a-a8e3-4722-8da0-a55911b4ac47","Type":"ContainerStarted","Data":"bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d"}
Jul 15 12:29:39.188016 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:39.188001    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:39.212985 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:39.212938    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7" podStartSLOduration=1.64643224 podStartE2EDuration="4.212923452s" totalImagesPullingTime="2.566491212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:39.210693011 +0000 UTC m=+568.087416017" watchObservedRunningTime="2026-07-15 12:29:39.212923452 +0000 UTC m=+568.089646458"
Jul 15 12:29:39.246153 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:39.246108    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987" podStartSLOduration=2.484925087 podStartE2EDuration="5.246092902s" totalImagesPullingTime="2.761167815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:29:39.245577059 +0000 UTC m=+568.122300066" watchObservedRunningTime="2026-07-15 12:29:39.246092902 +0000 UTC m=+568.122815907"
Jul 15 12:29:50.194637 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:50.194609    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:50.195101 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:50.194672    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:29:50.275463 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:50.275435    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c4f9ff7b7-7h987"]
Jul 15 12:29:50.275638 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:50.275609    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987" podUID="9defcc0a-a8e3-4722-8da0-a55911b4ac47" containerName="manager" containerID="cri-o://bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d" gracePeriod=30
Jul 15 12:29:51.054091 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.054067    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:51.167316 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.167237    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pr6r\" (UniqueName: \"kubernetes.io/projected/9defcc0a-a8e3-4722-8da0-a55911b4ac47-kube-api-access-2pr6r\") pod \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") "
Jul 15 12:29:51.167316 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.167298    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9defcc0a-a8e3-4722-8da0-a55911b4ac47-webhook-cert\") pod \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\" (UID: \"9defcc0a-a8e3-4722-8da0-a55911b4ac47\") "
Jul 15 12:29:51.169321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.169292    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9defcc0a-a8e3-4722-8da0-a55911b4ac47-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9defcc0a-a8e3-4722-8da0-a55911b4ac47" (UID: "9defcc0a-a8e3-4722-8da0-a55911b4ac47"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:29:51.169321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.169304    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9defcc0a-a8e3-4722-8da0-a55911b4ac47-kube-api-access-2pr6r" (OuterVolumeSpecName: "kube-api-access-2pr6r") pod "9defcc0a-a8e3-4722-8da0-a55911b4ac47" (UID: "9defcc0a-a8e3-4722-8da0-a55911b4ac47"). InnerVolumeSpecName "kube-api-access-2pr6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:29:51.243684 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.243657    2576 generic.go:356] "Generic (PLEG): container finished" podID="9defcc0a-a8e3-4722-8da0-a55911b4ac47" containerID="bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d" exitCode=0
Jul 15 12:29:51.244013 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.243697    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987"
Jul 15 12:29:51.244013 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.243737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987" event={"ID":"9defcc0a-a8e3-4722-8da0-a55911b4ac47","Type":"ContainerDied","Data":"bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d"}
Jul 15 12:29:51.244013 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.243788    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c4f9ff7b7-7h987" event={"ID":"9defcc0a-a8e3-4722-8da0-a55911b4ac47","Type":"ContainerDied","Data":"e0f64301afe78c9ee49d003799c99125e609fc532623d69872d806c3de7e77fe"}
Jul 15 12:29:51.244013 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.243809    2576 scope.go:117] "RemoveContainer" containerID="bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d"
Jul 15 12:29:51.252439 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.252423    2576 scope.go:117] "RemoveContainer" containerID="bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d"
Jul 15 12:29:51.252697 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:29:51.252678    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d\": container with ID starting with bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d not found: ID does not exist" containerID="bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d"
Jul 15 12:29:51.252766 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.252706    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d"} err="failed to get container status \"bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d\": rpc error: code = NotFound desc = could not find container \"bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d\": container with ID starting with bf0389d16ce98ec9695eff2050827e803a6e2a4c55674334e12fff60ea5de53d not found: ID does not exist"
Jul 15 12:29:51.268035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.268015    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c4f9ff7b7-7h987"]
Jul 15 12:29:51.268356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.268336    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pr6r\" (UniqueName: \"kubernetes.io/projected/9defcc0a-a8e3-4722-8da0-a55911b4ac47-kube-api-access-2pr6r\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:29:51.268451 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.268358    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9defcc0a-a8e3-4722-8da0-a55911b4ac47-webhook-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:29:51.272568 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.272523    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c4f9ff7b7-7h987"]
Jul 15 12:29:51.572059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:29:51.571988    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9defcc0a-a8e3-4722-8da0-a55911b4ac47" path="/var/lib/kubelet/pods/9defcc0a-a8e3-4722-8da0-a55911b4ac47/volumes"
Jul 15 12:30:11.071132 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.071098    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"]
Jul 15 12:30:11.071609 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.071480    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9defcc0a-a8e3-4722-8da0-a55911b4ac47" containerName="manager"
Jul 15 12:30:11.071609 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.071492    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9defcc0a-a8e3-4722-8da0-a55911b4ac47" containerName="manager"
Jul 15 12:30:11.071609 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.071568    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9defcc0a-a8e3-4722-8da0-a55911b4ac47" containerName="manager"
Jul 15 12:30:11.076206 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.076189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.080183 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.080160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 15 12:30:11.081246 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.081226    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rj5v4\""
Jul 15 12:30:11.113503 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.113482    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"]
Jul 15 12:30:11.227344 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.227311    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7801fd-6c76-4895-9307-667d6fcd92fa-openshift-service-ca\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.227344 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.227344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5n7p\" (UniqueName: \"kubernetes.io/projected/3e7801fd-6c76-4895-9307-667d6fcd92fa-kube-api-access-k5n7p\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.227526 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.227390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7801fd-6c76-4895-9307-667d6fcd92fa-maas-api-tls\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.328308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.328238    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7801fd-6c76-4895-9307-667d6fcd92fa-openshift-service-ca\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.328308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.328272    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5n7p\" (UniqueName: \"kubernetes.io/projected/3e7801fd-6c76-4895-9307-667d6fcd92fa-kube-api-access-k5n7p\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.328466 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.328313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7801fd-6c76-4895-9307-667d6fcd92fa-maas-api-tls\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.328861 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.328840    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e7801fd-6c76-4895-9307-667d6fcd92fa-openshift-service-ca\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.330715 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.330692    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e7801fd-6c76-4895-9307-667d6fcd92fa-maas-api-tls\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.341614 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.341595    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5n7p\" (UniqueName: \"kubernetes.io/projected/3e7801fd-6c76-4895-9307-667d6fcd92fa-kube-api-access-k5n7p\") pod \"maas-api-848c5c555b-bnrb7\" (UID: \"3e7801fd-6c76-4895-9307-667d6fcd92fa\") " pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.385734 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.385711    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:11.474785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.474749    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:30:11.476127 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.476104    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:30:11.532631 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:11.532604    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"]
Jul 15 12:30:11.535665 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:30:11.535640    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e7801fd_6c76_4895_9307_667d6fcd92fa.slice/crio-f6eff68f06b21a8bf60ce73e872a6b510e8e3369270dd15d73da995169e7d320 WatchSource:0}: Error finding container f6eff68f06b21a8bf60ce73e872a6b510e8e3369270dd15d73da995169e7d320: Status 404 returned error can't find the container with id f6eff68f06b21a8bf60ce73e872a6b510e8e3369270dd15d73da995169e7d320
Jul 15 12:30:12.208844 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.208811    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"]
Jul 15 12:30:12.212893 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.212874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.215799 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.215776    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bfqhx\""
Jul 15 12:30:12.216082 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.216056    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 15 12:30:12.229119 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.229092    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"]
Jul 15 12:30:12.236293 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.236266    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74d4e7a1-6b77-492b-8d7c-a1aaaef49046-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-w8vs9\" (UID: \"74d4e7a1-6b77-492b-8d7c-a1aaaef49046\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.236401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.236344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67qk\" (UniqueName: \"kubernetes.io/projected/74d4e7a1-6b77-492b-8d7c-a1aaaef49046-kube-api-access-k67qk\") pod \"payload-pre-processing-74584bb9f8-w8vs9\" (UID: \"74d4e7a1-6b77-492b-8d7c-a1aaaef49046\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.338100 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.336995    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67qk\" (UniqueName: \"kubernetes.io/projected/74d4e7a1-6b77-492b-8d7c-a1aaaef49046-kube-api-access-k67qk\") pod \"payload-pre-processing-74584bb9f8-w8vs9\" (UID: \"74d4e7a1-6b77-492b-8d7c-a1aaaef49046\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.338100 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.337105    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74d4e7a1-6b77-492b-8d7c-a1aaaef49046-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-w8vs9\" (UID: \"74d4e7a1-6b77-492b-8d7c-a1aaaef49046\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.338100 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.338003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74d4e7a1-6b77-492b-8d7c-a1aaaef49046-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-w8vs9\" (UID: \"74d4e7a1-6b77-492b-8d7c-a1aaaef49046\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.341267 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.341237    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7" event={"ID":"3e7801fd-6c76-4895-9307-667d6fcd92fa","Type":"ContainerStarted","Data":"f6eff68f06b21a8bf60ce73e872a6b510e8e3369270dd15d73da995169e7d320"}
Jul 15 12:30:12.346868 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.346841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67qk\" (UniqueName: \"kubernetes.io/projected/74d4e7a1-6b77-492b-8d7c-a1aaaef49046-kube-api-access-k67qk\") pod \"payload-pre-processing-74584bb9f8-w8vs9\" (UID: \"74d4e7a1-6b77-492b-8d7c-a1aaaef49046\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.524076 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.523996    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:12.705727 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:12.705702    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"]
Jul 15 12:30:12.708185 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:30:12.708154    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d4e7a1_6b77_492b_8d7c_a1aaaef49046.slice/crio-3609ad26d7ab405cb165bf2ebb6618502a7b6290548a5d913218b52c03df6858 WatchSource:0}: Error finding container 3609ad26d7ab405cb165bf2ebb6618502a7b6290548a5d913218b52c03df6858: Status 404 returned error can't find the container with id 3609ad26d7ab405cb165bf2ebb6618502a7b6290548a5d913218b52c03df6858
Jul 15 12:30:13.353209 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:13.353172    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9" event={"ID":"74d4e7a1-6b77-492b-8d7c-a1aaaef49046","Type":"ContainerStarted","Data":"3609ad26d7ab405cb165bf2ebb6618502a7b6290548a5d913218b52c03df6858"}
Jul 15 12:30:14.359355 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:14.359317    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7" event={"ID":"3e7801fd-6c76-4895-9307-667d6fcd92fa","Type":"ContainerStarted","Data":"55d8d105688fbc7dee0a8d843c0cc4035c368ba85bb36a04814dbc7a3f3272a9"}
Jul 15 12:30:14.359794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:14.359467    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:14.379211 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:14.379161    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7" podStartSLOduration=1.5631698950000001 podStartE2EDuration="3.37914576s" totalImagesPullingTime="1.815975865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:14.378393048 +0000 UTC m=+603.255116055" watchObservedRunningTime="2026-07-15 12:30:14.37914576 +0000 UTC m=+603.255868766"
Jul 15 12:30:17.375038 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:17.374995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9" event={"ID":"74d4e7a1-6b77-492b-8d7c-a1aaaef49046","Type":"ContainerStarted","Data":"69187a158444c0201fd712f27be810ef87e2368a53edeac437ddd204fd911384"}
Jul 15 12:30:17.375523 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:17.375072    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:19.385607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.385582    2576 generic.go:356] "Generic (PLEG): container finished" podID="f6e88f17-42ee-47aa-b377-572cee4dcb0f" containerID="7420675cea1b40785a7a244ca03551855899281d801c8f44ff22f311c9391e7d" exitCode=0
Jul 15 12:30:19.385914 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.385650    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" event={"ID":"f6e88f17-42ee-47aa-b377-572cee4dcb0f","Type":"ContainerDied","Data":"7420675cea1b40785a7a244ca03551855899281d801c8f44ff22f311c9391e7d"}
Jul 15 12:30:19.386035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.386022    2576 scope.go:117] "RemoveContainer" containerID="7420675cea1b40785a7a244ca03551855899281d801c8f44ff22f311c9391e7d"
Jul 15 12:30:19.386991 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.386965    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:30:19.387047 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.387029    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:30:19.387085 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.387057    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:30:19.455165 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:19.455119    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9" podStartSLOduration=3.450711464 podStartE2EDuration="7.455103319s" totalImagesPullingTime="4.004391855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:17.415954219 +0000 UTC m=+606.292677224" watchObservedRunningTime="2026-07-15 12:30:19.455103319 +0000 UTC m=+608.331826319"
Jul 15 12:30:20.370331 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:20.370296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-848c5c555b-bnrb7"
Jul 15 12:30:20.395606 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:20.395568    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" event={"ID":"f6e88f17-42ee-47aa-b377-572cee4dcb0f","Type":"ContainerStarted","Data":"5f5332a1df4233395fade5558ac7cac6fec4246aff7797964055b01504faac8c"}
Jul 15 12:30:20.625001 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:20.624898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:30:20.626051 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:20.626022    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 15 12:30:20.626271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:20.626239    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" podUID="f6e88f17-42ee-47aa-b377-572cee4dcb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 15 12:30:20.931036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:20.930993    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:30:21.625319 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:21.625285    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 15 12:30:21.625801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:21.625345    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" podUID="f6e88f17-42ee-47aa-b377-572cee4dcb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 15 12:30:22.625648 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:22.625609    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 15 12:30:22.626098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:22.625672    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" podUID="f6e88f17-42ee-47aa-b377-572cee4dcb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 15 12:30:23.625981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:23.625579    2576 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-kfflk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 15 12:30:23.625981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:23.625635    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk" podUID="f6e88f17-42ee-47aa-b377-572cee4dcb0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 15 12:30:24.629516 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:24.629488    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:30:24.630552 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:24.630518    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-kfflk"
Jul 15 12:30:26.097364 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.097334    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"]
Jul 15 12:30:26.102020 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.102001    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.104671 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.104651    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 15 12:30:26.104824 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.104809    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 15 12:30:26.104897 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.104812    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f96dj\""
Jul 15 12:30:26.105488 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.105471    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 15 12:30:26.115096 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.115077    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"]
Jul 15 12:30:26.172101 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.172076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.172219 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.172105    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61aacc84-4e1a-460f-b489-0854e26cfe91-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.172219 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.172125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.172219 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.172139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hf6s\" (UniqueName: \"kubernetes.io/projected/61aacc84-4e1a-460f-b489-0854e26cfe91-kube-api-access-4hf6s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.172320 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.172212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.172320 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.172272    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.273603 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.273575    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.273729 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.273621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.273729 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.273676    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.273729 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.273692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61aacc84-4e1a-460f-b489-0854e26cfe91-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.273729 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.273709    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.273915 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.273731    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hf6s\" (UniqueName: \"kubernetes.io/projected/61aacc84-4e1a-460f-b489-0854e26cfe91-kube-api-access-4hf6s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.274025 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.274003    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.274086 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.274018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.274086 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.274063    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.275930 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.275909    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61aacc84-4e1a-460f-b489-0854e26cfe91-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.276295 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.276275    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61aacc84-4e1a-460f-b489-0854e26cfe91-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.282201 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.282179    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hf6s\" (UniqueName: \"kubernetes.io/projected/61aacc84-4e1a-460f-b489-0854e26cfe91-kube-api-access-4hf6s\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh\" (UID: \"61aacc84-4e1a-460f-b489-0854e26cfe91\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.412238 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.412211    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:26.547511 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:26.547478    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"]
Jul 15 12:30:26.548889 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:30:26.548862    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61aacc84_4e1a_460f_b489_0854e26cfe91.slice/crio-1fbc67c2d78189ac81800f2895abb5fa0c494a342bac68e8e4a7abfae39d7ade WatchSource:0}: Error finding container 1fbc67c2d78189ac81800f2895abb5fa0c494a342bac68e8e4a7abfae39d7ade: Status 404 returned error can't find the container with id 1fbc67c2d78189ac81800f2895abb5fa0c494a342bac68e8e4a7abfae39d7ade
Jul 15 12:30:27.428790 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:27.428749    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh" event={"ID":"61aacc84-4e1a-460f-b489-0854e26cfe91","Type":"ContainerStarted","Data":"1fbc67c2d78189ac81800f2895abb5fa0c494a342bac68e8e4a7abfae39d7ade"}
Jul 15 12:30:28.382307 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:28.382263    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9" podUID="74d4e7a1-6b77-492b-8d7c-a1aaaef49046" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.40:9004: connect: connection refused"
Jul 15 12:30:28.435010 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:28.434974    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh" event={"ID":"61aacc84-4e1a-460f-b489-0854e26cfe91","Type":"ContainerStarted","Data":"3aaac6ed8133fb4ef6f927df5547448fc5c0a828ee94ade06ece9aa0b8b66488"}
Jul 15 12:30:28.469720 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:28.469674    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh" podStartSLOduration=1.285377255 podStartE2EDuration="2.469659841s" totalImagesPullingTime="1.184282586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:28.468515806 +0000 UTC m=+617.345238811" watchObservedRunningTime="2026-07-15 12:30:28.469659841 +0000 UTC m=+617.346382846"
Jul 15 12:30:36.415748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.415650    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:36.416088 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.415777    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:36.428023 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.427999    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:36.481243 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.481222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh"
Jul 15 12:30:36.726585 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.726510    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"]
Jul 15 12:30:36.729801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.729786    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.753349 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.753328    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 15 12:30:36.759151 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.759127    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"]
Jul 15 12:30:36.859980 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.859953    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.860116 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.859985    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.860116 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.860011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.860116 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.860044    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlckf\" (UniqueName: \"kubernetes.io/projected/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-kube-api-access-mlckf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.860229 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.860116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.860229 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.860138    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.960746 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.960715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.960746 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.960749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.960935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.960774    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.960935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.960797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlckf\" (UniqueName: \"kubernetes.io/projected/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-kube-api-access-mlckf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.960935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.960868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.960935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.960893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.961227 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.961203    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.961305 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.961243    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.961356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.961312    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.962935 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.962914    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.963100 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.963084    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:36.972785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:36.972764    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlckf\" (UniqueName: \"kubernetes.io/projected/d1d611ca-1c1c-4500-9c37-aeb00b3675f7-kube-api-access-mlckf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-xc4np\" (UID: \"d1d611ca-1c1c-4500-9c37-aeb00b3675f7\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:37.039456 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:37.039391    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:37.193649 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:37.193494    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"]
Jul 15 12:30:37.196139 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:30:37.196116    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d611ca_1c1c_4500_9c37_aeb00b3675f7.slice/crio-b097716b24d99572eddc680fab8bb8facc2e7310abaee4e468a762eae9c98eb5 WatchSource:0}: Error finding container b097716b24d99572eddc680fab8bb8facc2e7310abaee4e468a762eae9c98eb5: Status 404 returned error can't find the container with id b097716b24d99572eddc680fab8bb8facc2e7310abaee4e468a762eae9c98eb5
Jul 15 12:30:37.475438 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:37.475406    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np" event={"ID":"d1d611ca-1c1c-4500-9c37-aeb00b3675f7","Type":"ContainerStarted","Data":"1db0e4bb42de7d72b4cf6df3a1100c44cf6ee6a70f974ce4040c69373f990aa2"}
Jul 15 12:30:37.475831 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:37.475458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np" event={"ID":"d1d611ca-1c1c-4500-9c37-aeb00b3675f7","Type":"ContainerStarted","Data":"b097716b24d99572eddc680fab8bb8facc2e7310abaee4e468a762eae9c98eb5"}
Jul 15 12:30:37.514787 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:37.514603    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np" podStartSLOduration=1.3248021269999999 podStartE2EDuration="1.514585014s" totalImagesPullingTime="189.782887ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:37.509931479 +0000 UTC m=+626.386654484" watchObservedRunningTime="2026-07-15 12:30:37.514585014 +0000 UTC m=+626.391308020"
Jul 15 12:30:38.380874 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:38.380845    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-w8vs9"
Jul 15 12:30:46.631415 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.631387    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"]
Jul 15 12:30:46.637435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.637419    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.641028 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.641004    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 15 12:30:46.728344 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.728307    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"]
Jul 15 12:30:46.749572 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.749543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.749700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.749621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.749700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.749663    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.749700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.749683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.749854 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.749704    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr6sg\" (UniqueName: \"kubernetes.io/projected/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-kube-api-access-cr6sg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.749854 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.749826    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.850793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.850764    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.850924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.850813    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.850924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.850851    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.850924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.850870    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.850924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.850885    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.850924 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.850907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr6sg\" (UniqueName: \"kubernetes.io/projected/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-kube-api-access-cr6sg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.851208 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.851183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.851276 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.851206    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.851276 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.851234    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.853109 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.853093    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.853357 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.853339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.868827 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.868807    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr6sg\" (UniqueName: \"kubernetes.io/projected/8f87f326-b03f-45ec-b7ba-1dfa66637fa9-kube-api-access-cr6sg\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq\" (UID: \"8f87f326-b03f-45ec-b7ba-1dfa66637fa9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:46.948340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:46.948313    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:47.039789 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.039754    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:47.039933 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.039837    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:47.053719 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.053693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:47.116951 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.116927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"]
Jul 15 12:30:47.118978 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:30:47.118953    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f87f326_b03f_45ec_b7ba_1dfa66637fa9.slice/crio-fb9abf280b01801c7d30dd6f0fbe546b31cdedffddd30c8726972ed3e4b5bc4e WatchSource:0}: Error finding container fb9abf280b01801c7d30dd6f0fbe546b31cdedffddd30c8726972ed3e4b5bc4e: Status 404 returned error can't find the container with id fb9abf280b01801c7d30dd6f0fbe546b31cdedffddd30c8726972ed3e4b5bc4e
Jul 15 12:30:47.120267 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.120249    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:30:47.527146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.527111    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq" event={"ID":"8f87f326-b03f-45ec-b7ba-1dfa66637fa9","Type":"ContainerStarted","Data":"d6f099c0335770643e54f6cd5ac797960ddd3b2395453bfc7dfb4a69ff91dcb5"}
Jul 15 12:30:47.527336 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.527189    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq" event={"ID":"8f87f326-b03f-45ec-b7ba-1dfa66637fa9","Type":"ContainerStarted","Data":"fb9abf280b01801c7d30dd6f0fbe546b31cdedffddd30c8726972ed3e4b5bc4e"}
Jul 15 12:30:47.538175 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.538150    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-xc4np"
Jul 15 12:30:47.622648 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:47.622561    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq" podStartSLOduration=1.425667058 podStartE2EDuration="1.62254673s" totalImagesPullingTime="196.879672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:30:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:30:47.576593061 +0000 UTC m=+636.453316067" watchObservedRunningTime="2026-07-15 12:30:47.62254673 +0000 UTC m=+636.499269726"
Jul 15 12:30:56.949284 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:56.949247    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:56.949709 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:56.949321    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:56.961416 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:56.961393    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:30:57.581951 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:30:57.581918    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq"
Jul 15 12:34:08.805273 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:08.805243    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-h9bb7"]
Jul 15 12:34:08.805800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:08.805429    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7" podUID="a0c8e0a3-d7ff-4f61-853b-39c06407b827" containerName="manager" containerID="cri-o://0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d" gracePeriod=30
Jul 15 12:34:09.058382 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.058323    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:34:09.187369 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.187323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c8e0a3-d7ff-4f61-853b-39c06407b827-webhook-cert\") pod \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") "
Jul 15 12:34:09.187581 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.187386    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdbcp\" (UniqueName: \"kubernetes.io/projected/a0c8e0a3-d7ff-4f61-853b-39c06407b827-kube-api-access-wdbcp\") pod \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\" (UID: \"a0c8e0a3-d7ff-4f61-853b-39c06407b827\") "
Jul 15 12:34:09.189321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.189290    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0c8e0a3-d7ff-4f61-853b-39c06407b827-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a0c8e0a3-d7ff-4f61-853b-39c06407b827" (UID: "a0c8e0a3-d7ff-4f61-853b-39c06407b827"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:34:09.189430 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.189335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c8e0a3-d7ff-4f61-853b-39c06407b827-kube-api-access-wdbcp" (OuterVolumeSpecName: "kube-api-access-wdbcp") pod "a0c8e0a3-d7ff-4f61-853b-39c06407b827" (UID: "a0c8e0a3-d7ff-4f61-853b-39c06407b827"). InnerVolumeSpecName "kube-api-access-wdbcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:34:09.289066 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.289035    2576 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0c8e0a3-d7ff-4f61-853b-39c06407b827-webhook-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:34:09.289066 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.289060    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdbcp\" (UniqueName: \"kubernetes.io/projected/a0c8e0a3-d7ff-4f61-853b-39c06407b827-kube-api-access-wdbcp\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:34:09.445763 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.445724    2576 generic.go:356] "Generic (PLEG): container finished" podID="a0c8e0a3-d7ff-4f61-853b-39c06407b827" containerID="0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d" exitCode=0
Jul 15 12:34:09.445944 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.445774    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7"
Jul 15 12:34:09.445944 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.445789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7" event={"ID":"a0c8e0a3-d7ff-4f61-853b-39c06407b827","Type":"ContainerDied","Data":"0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d"}
Jul 15 12:34:09.445944 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.445841    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fcfc6ff5-h9bb7" event={"ID":"a0c8e0a3-d7ff-4f61-853b-39c06407b827","Type":"ContainerDied","Data":"5569b1307cfd5d986fdec99796f28180a3ac1724e7b788d702f5a051060abca9"}
Jul 15 12:34:09.445944 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.445873    2576 scope.go:117] "RemoveContainer" containerID="0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d"
Jul 15 12:34:09.454953 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.454935    2576 scope.go:117] "RemoveContainer" containerID="0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d"
Jul 15 12:34:09.455202 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:34:09.455186    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d\": container with ID starting with 0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d not found: ID does not exist" containerID="0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d"
Jul 15 12:34:09.455254 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.455210    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d"} err="failed to get container status \"0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d\": rpc error: code = NotFound desc = could not find container \"0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d\": container with ID starting with 0b0152a97265a603c35d040c85f0cad26d919f08e464bd53e5afa27de817158d not found: ID does not exist"
Jul 15 12:34:09.475331 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.475296    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-h9bb7"]
Jul 15 12:34:09.480301 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.480273    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-h9bb7"]
Jul 15 12:34:09.572363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:09.572335    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0c8e0a3-d7ff-4f61-853b-39c06407b827" path="/var/lib/kubelet/pods/a0c8e0a3-d7ff-4f61-853b-39c06407b827/volumes"
Jul 15 12:34:10.342889 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.342853    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-jr8kd"]
Jul 15 12:34:10.343266 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.343211    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c8e0a3-d7ff-4f61-853b-39c06407b827" containerName="manager"
Jul 15 12:34:10.343266 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.343221    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c8e0a3-d7ff-4f61-853b-39c06407b827" containerName="manager"
Jul 15 12:34:10.349526 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.343710    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c8e0a3-d7ff-4f61-853b-39c06407b827" containerName="manager"
Jul 15 12:34:10.349526 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.349172    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.352685 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.352665    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 12:34:10.353511 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.353491    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 15 12:34:10.353959 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.353945    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-4qlgc\""
Jul 15 12:34:10.378700 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.378670    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-jr8kd"]
Jul 15 12:34:10.499483 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.499450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfjcm\" (UniqueName: \"kubernetes.io/projected/fdd0db81-5abc-41a8-9abc-5d01af1501cb-kube-api-access-tfjcm\") pod \"maas-controller-74fcfc6ff5-jr8kd\" (UID: \"fdd0db81-5abc-41a8-9abc-5d01af1501cb\") " pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.499684 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.499504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fdd0db81-5abc-41a8-9abc-5d01af1501cb-webhook-cert\") pod \"maas-controller-74fcfc6ff5-jr8kd\" (UID: \"fdd0db81-5abc-41a8-9abc-5d01af1501cb\") " pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.600271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.600184    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfjcm\" (UniqueName: \"kubernetes.io/projected/fdd0db81-5abc-41a8-9abc-5d01af1501cb-kube-api-access-tfjcm\") pod \"maas-controller-74fcfc6ff5-jr8kd\" (UID: \"fdd0db81-5abc-41a8-9abc-5d01af1501cb\") " pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.600271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.600244    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fdd0db81-5abc-41a8-9abc-5d01af1501cb-webhook-cert\") pod \"maas-controller-74fcfc6ff5-jr8kd\" (UID: \"fdd0db81-5abc-41a8-9abc-5d01af1501cb\") " pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.602791 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.602764    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fdd0db81-5abc-41a8-9abc-5d01af1501cb-webhook-cert\") pod \"maas-controller-74fcfc6ff5-jr8kd\" (UID: \"fdd0db81-5abc-41a8-9abc-5d01af1501cb\") " pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.633893 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.633857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfjcm\" (UniqueName: \"kubernetes.io/projected/fdd0db81-5abc-41a8-9abc-5d01af1501cb-kube-api-access-tfjcm\") pod \"maas-controller-74fcfc6ff5-jr8kd\" (UID: \"fdd0db81-5abc-41a8-9abc-5d01af1501cb\") " pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.660607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.660574    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:10.800856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:10.800830    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74fcfc6ff5-jr8kd"]
Jul 15 12:34:10.802428 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:34:10.802400    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdd0db81_5abc_41a8_9abc_5d01af1501cb.slice/crio-e435f3ccecc1a832bacacb237cc8f3aa673fa6ffafe4532b9ca9d5240d0d260a WatchSource:0}: Error finding container e435f3ccecc1a832bacacb237cc8f3aa673fa6ffafe4532b9ca9d5240d0d260a: Status 404 returned error can't find the container with id e435f3ccecc1a832bacacb237cc8f3aa673fa6ffafe4532b9ca9d5240d0d260a
Jul 15 12:34:11.456624 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:11.456590    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd" event={"ID":"fdd0db81-5abc-41a8-9abc-5d01af1501cb","Type":"ContainerStarted","Data":"e435f3ccecc1a832bacacb237cc8f3aa673fa6ffafe4532b9ca9d5240d0d260a"}
Jul 15 12:34:15.388757 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:15.388735    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 15 12:34:15.478315 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:15.478270    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd" event={"ID":"fdd0db81-5abc-41a8-9abc-5d01af1501cb","Type":"ContainerStarted","Data":"c5d01134816cf05a412d176280ef6a275e6d59dbc1737a8fb10d923f3ecb5769"}
Jul 15 12:34:15.478501 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:15.478433    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:34:15.499295 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:15.499242    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd" podStartSLOduration=0.916792705 podStartE2EDuration="5.499224861s" totalImagesPullingTime="4.582432156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:34:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:34:15.498480729 +0000 UTC m=+844.375203728" watchObservedRunningTime="2026-07-15 12:34:15.499224861 +0000 UTC m=+844.375947893"
Jul 15 12:34:26.486809 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:34:26.486775    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74fcfc6ff5-jr8kd"
Jul 15 12:35:11.505767 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:35:11.505692    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:35:11.508792 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:35:11.508769    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:40:11.532512 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:40:11.532423    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:40:11.536106 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:40:11.536086    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:45:00.153595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.153565    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"]
Jul 15 12:45:00.156735 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.156716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"
Jul 15 12:45:00.162018 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.161994    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-267b6\""
Jul 15 12:45:00.165693 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.165677    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"
Jul 15 12:45:00.189641 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.189606    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"]
Jul 15 12:45:00.301850 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.301824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"]
Jul 15 12:45:00.303229 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:45:00.303206    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fda6ad1_c204_4f28_81e5_61c149063bf8.slice/crio-227d8070cfcfba42a2e224870bc7832a06934990ff602e38e894a7306409e531 WatchSource:0}: Error finding container 227d8070cfcfba42a2e224870bc7832a06934990ff602e38e894a7306409e531: Status 404 returned error can't find the container with id 227d8070cfcfba42a2e224870bc7832a06934990ff602e38e894a7306409e531
Jul 15 12:45:00.304995 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.304979    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:45:00.406840 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:00.406774    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4" event={"ID":"9fda6ad1-c204-4f28-81e5-61c149063bf8","Type":"ContainerStarted","Data":"227d8070cfcfba42a2e224870bc7832a06934990ff602e38e894a7306409e531"}
Jul 15 12:45:03.423990 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:03.423950    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4" event={"ID":"9fda6ad1-c204-4f28-81e5-61c149063bf8","Type":"ContainerStarted","Data":"7dc15c93c79e5f191dd0a164f98dd2485e30a44fa4cbc7de772752007cdbce33"}
Jul 15 12:45:03.439410 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:03.439357    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4" podStartSLOduration=0.839458978 podStartE2EDuration="3.439342505s" totalImagesPullingTime="2.599883527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:45:03.438780336 +0000 UTC m=+1492.315503338" watchObservedRunningTime="2026-07-15 12:45:03.439342505 +0000 UTC m=+1492.316065511"
Jul 15 12:45:04.429474 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:04.429443    2576 generic.go:356] "Generic (PLEG): container finished" podID="9fda6ad1-c204-4f28-81e5-61c149063bf8" containerID="7dc15c93c79e5f191dd0a164f98dd2485e30a44fa4cbc7de772752007cdbce33" exitCode=0
Jul 15 12:45:04.429872 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:04.429551    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4" event={"ID":"9fda6ad1-c204-4f28-81e5-61c149063bf8","Type":"ContainerDied","Data":"7dc15c93c79e5f191dd0a164f98dd2485e30a44fa4cbc7de772752007cdbce33"}
Jul 15 12:45:05.562509 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:05.562481    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"
Jul 15 12:45:06.440282 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:06.440255    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4"
Jul 15 12:45:06.440471 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:06.440299    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735325-sp5p4" event={"ID":"9fda6ad1-c204-4f28-81e5-61c149063bf8","Type":"ContainerDied","Data":"227d8070cfcfba42a2e224870bc7832a06934990ff602e38e894a7306409e531"}
Jul 15 12:45:06.440471 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:06.440344    2576 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="227d8070cfcfba42a2e224870bc7832a06934990ff602e38e894a7306409e531"
Jul 15 12:45:11.560628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:11.560593    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:45:11.565228 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:45:11.565207    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:46:19.469386 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.469352    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"]
Jul 15 12:46:19.469903 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.469888    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fda6ad1-c204-4f28-81e5-61c149063bf8" containerName="cleanup"
Jul 15 12:46:19.469947 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.469906    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fda6ad1-c204-4f28-81e5-61c149063bf8" containerName="cleanup"
Jul 15 12:46:19.470031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.470011    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fda6ad1-c204-4f28-81e5-61c149063bf8" containerName="cleanup"
Jul 15 12:46:19.473253 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.473233    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.475745 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.475724    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2bbbbc27-serving-cert\""
Jul 15 12:46:19.485060 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.485037    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"]
Jul 15 12:46:19.571146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.571112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.571310 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.571164    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxpdp\" (UniqueName: \"kubernetes.io/projected/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-kube-api-access-kxpdp\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.571310 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.571215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-openshift-service-ca\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.672649 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.672617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-openshift-service-ca\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.672773 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.672685    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.672773 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.672738    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxpdp\" (UniqueName: \"kubernetes.io/projected/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-kube-api-access-kxpdp\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.672845 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:46:19.672786    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-serving-cert: secret "maas-api-e2e-ait-2bbbbc27-serving-cert" not found
Jul 15 12:46:19.672877 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:46:19.672850    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls podName:9e9994e8-32d9-4dbf-901e-548ec1bbd5f1 nodeName:}" failed. No retries permitted until 2026-07-15 12:46:20.172832404 +0000 UTC m=+1569.049555406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls") pod "maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" (UID: "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1") : secret "maas-api-e2e-ait-2bbbbc27-serving-cert" not found
Jul 15 12:46:19.673358 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.673334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-openshift-service-ca\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:19.690666 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:19.690642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxpdp\" (UniqueName: \"kubernetes.io/projected/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-kube-api-access-kxpdp\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:20.176395 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:20.176360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:20.178862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:20.178841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls\") pod \"maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:20.384012 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:20.383976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:20.729037 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:20.729013    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"]
Jul 15 12:46:20.730683 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:46:20.730656    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e9994e8_32d9_4dbf_901e_548ec1bbd5f1.slice/crio-66566728935a8aa2d661862ab4ebd3734fb4ed0e4f7f66dd13ff1f7bd34805e8 WatchSource:0}: Error finding container 66566728935a8aa2d661862ab4ebd3734fb4ed0e4f7f66dd13ff1f7bd34805e8: Status 404 returned error can't find the container with id 66566728935a8aa2d661862ab4ebd3734fb4ed0e4f7f66dd13ff1f7bd34805e8
Jul 15 12:46:20.783174 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:20.783147    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" event={"ID":"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1","Type":"ContainerStarted","Data":"66566728935a8aa2d661862ab4ebd3734fb4ed0e4f7f66dd13ff1f7bd34805e8"}
Jul 15 12:46:21.795888 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:21.795804    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" event={"ID":"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1","Type":"ContainerStarted","Data":"d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9"}
Jul 15 12:46:21.796218 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:21.795941    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:21.854865 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:21.854821    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" podStartSLOduration=2.177992179 podStartE2EDuration="2.854805974s" totalImagesPullingTime="676.813795ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:46:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:46:21.853334406 +0000 UTC m=+1570.730057412" watchObservedRunningTime="2026-07-15 12:46:21.854805974 +0000 UTC m=+1570.731528980"
Jul 15 12:46:27.806334 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:27.806223    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:35.131251 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.131222    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"]
Jul 15 12:46:35.131674 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.131442    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" podUID="9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" containerName="maas-api" containerID="cri-o://d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9" gracePeriod=30
Jul 15 12:46:35.378072 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.378050    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:35.506616 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.506587    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls\") pod \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") "
Jul 15 12:46:35.506766 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.506679    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxpdp\" (UniqueName: \"kubernetes.io/projected/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-kube-api-access-kxpdp\") pod \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") "
Jul 15 12:46:35.506766 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.506742    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-openshift-service-ca\") pod \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\" (UID: \"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1\") "
Jul 15 12:46:35.507092 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.507063    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" (UID: "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:46:35.508793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.508772    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" (UID: "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:46:35.508904 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.508810    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-kube-api-access-kxpdp" (OuterVolumeSpecName: "kube-api-access-kxpdp") pod "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" (UID: "9e9994e8-32d9-4dbf-901e-548ec1bbd5f1"). InnerVolumeSpecName "kube-api-access-kxpdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:46:35.608280 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.608259    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxpdp\" (UniqueName: \"kubernetes.io/projected/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-kube-api-access-kxpdp\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:46:35.608280 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.608279    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:46:35.608405 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.608289    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:46:35.864098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.864023    2576 generic.go:356] "Generic (PLEG): container finished" podID="9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" containerID="d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9" exitCode=0
Jul 15 12:46:35.864098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.864066    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" event={"ID":"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1","Type":"ContainerDied","Data":"d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9"}
Jul 15 12:46:35.864098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.864072    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"
Jul 15 12:46:35.864279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.864118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm" event={"ID":"9e9994e8-32d9-4dbf-901e-548ec1bbd5f1","Type":"ContainerDied","Data":"66566728935a8aa2d661862ab4ebd3734fb4ed0e4f7f66dd13ff1f7bd34805e8"}
Jul 15 12:46:35.864279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.864147    2576 scope.go:117] "RemoveContainer" containerID="d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9"
Jul 15 12:46:35.872510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.872492    2576 scope.go:117] "RemoveContainer" containerID="d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9"
Jul 15 12:46:35.872780 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:46:35.872761    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9\": container with ID starting with d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9 not found: ID does not exist" containerID="d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9"
Jul 15 12:46:35.872832 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.872790    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9"} err="failed to get container status \"d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9\": rpc error: code = NotFound desc = could not find container \"d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9\": container with ID starting with d95476e2edf8f977926d4ab8c0caede19a918b0e2d3438989e23b42f0a7b3ce9 not found: ID does not exist"
Jul 15 12:46:35.892485 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.892463    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"]
Jul 15 12:46:35.906717 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:35.906693    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2bbbbc27-6db5487f99-9fplm"]
Jul 15 12:46:37.572381 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:37.572346    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" path="/var/lib/kubelet/pods/9e9994e8-32d9-4dbf-901e-548ec1bbd5f1/volumes"
Jul 15 12:46:50.324181 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.324144    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"]
Jul 15 12:46:50.324947 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.324695    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" containerName="maas-api"
Jul 15 12:46:50.324947 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.324717    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" containerName="maas-api"
Jul 15 12:46:50.324947 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.324846    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e9994e8-32d9-4dbf-901e-548ec1bbd5f1" containerName="maas-api"
Jul 15 12:46:50.327996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.327976    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.331204 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.331173    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-907cfafc-gw-openshift-default-dockercfg-g496g\""
Jul 15 12:46:50.341403 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.341381    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"]
Jul 15 12:46:50.421358 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-socket\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-credential-socket\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421423    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-data\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-certs\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421500    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e863f53a-e258-4d23-bfed-75312a6f90c4-istiod-ca-cert\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421527    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5pck\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-kube-api-access-z5pck\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421591    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-podinfo\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421702 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421613    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-envoy\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.421702 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.421656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-token\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.522643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-podinfo\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.522643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-envoy\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.522862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522669    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-token\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.522862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522703    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-socket\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.522862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-credential-socket\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.522862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-data\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523156 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.522805    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-certs\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523156 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523147    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-credential-socket\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523270 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-data\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523270 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523236    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e863f53a-e258-4d23-bfed-75312a6f90c4-istiod-ca-cert\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523366 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523279    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5pck\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-kube-api-access-z5pck\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523466 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523439    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-certs\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523520 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523464    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-socket\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.523952 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.523931    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e863f53a-e258-4d23-bfed-75312a6f90c4-istiod-ca-cert\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.525466 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.525436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-envoy\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.525632 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.525617    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-podinfo\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.535709 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.535684    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-token\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.536978 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.536959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5pck\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-kube-api-access-z5pck\") pod \"e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") " pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.641925 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.641869    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:50.780712 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.780690    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"]
Jul 15 12:46:50.782459 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:46:50.782426    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode863f53a_e258_4d23_bfed_75312a6f90c4.slice/crio-cd7719bc726b25690d533585e9d2f200cda194d73950c97f09910b63a6fa0cf6 WatchSource:0}: Error finding container cd7719bc726b25690d533585e9d2f200cda194d73950c97f09910b63a6fa0cf6: Status 404 returned error can't find the container with id cd7719bc726b25690d533585e9d2f200cda194d73950c97f09910b63a6fa0cf6
Jul 15 12:46:50.784504 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.784470    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:46:50.784625 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.784547    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:46:50.784625 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.784576    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:46:50.945010 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.944978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" event={"ID":"e863f53a-e258-4d23-bfed-75312a6f90c4","Type":"ContainerStarted","Data":"5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"}
Jul 15 12:46:50.945213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.945042    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" event={"ID":"e863f53a-e258-4d23-bfed-75312a6f90c4","Type":"ContainerStarted","Data":"cd7719bc726b25690d533585e9d2f200cda194d73950c97f09910b63a6fa0cf6"}
Jul 15 12:46:50.969719 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:50.969675    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podStartSLOduration=0.969660236 podStartE2EDuration="969.660236ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:46:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:46:50.967288928 +0000 UTC m=+1599.844011934" watchObservedRunningTime="2026-07-15 12:46:50.969660236 +0000 UTC m=+1599.846383241"
Jul 15 12:46:51.642511 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.642482    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:51.643908 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.643884    2576 patch_prober.go:28] interesting pod/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 15 12:46:51.644017 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.643931    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 15 12:46:51.708932 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.708900    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"]
Jul 15 12:46:51.713145 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.713122    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.715470 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.715446    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-907cfafc-serving-cert\""
Jul 15 12:46:51.725552 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.725508    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"]
Jul 15 12:46:51.834637 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.834602    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-maas-api-tls\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.834819 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.834662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98q87\" (UniqueName: \"kubernetes.io/projected/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-kube-api-access-98q87\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.834819 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.834733    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-openshift-service-ca\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.935440 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.935413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-openshift-service-ca\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.935627 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.935469    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-maas-api-tls\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.935627 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.935501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98q87\" (UniqueName: \"kubernetes.io/projected/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-kube-api-access-98q87\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.936156 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.936134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-openshift-service-ca\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.937919 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.937890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-maas-api-tls\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:51.956469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:51.956448    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98q87\" (UniqueName: \"kubernetes.io/projected/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-kube-api-access-98q87\") pod \"maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:52.026043 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.026018    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:52.163076 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.163053    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"]
Jul 15 12:46:52.165061 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:46:52.165034    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fd3fb95_bff1_4efd_a9e8_aeed0517b4f2.slice/crio-b0df652896a857f31335d6a595699cf456cdfd179ba27c33925750cf24b29fa3 WatchSource:0}: Error finding container b0df652896a857f31335d6a595699cf456cdfd179ba27c33925750cf24b29fa3: Status 404 returned error can't find the container with id b0df652896a857f31335d6a595699cf456cdfd179ba27c33925750cf24b29fa3
Jul 15 12:46:52.646969 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.646945    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:52.965105 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.965064    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k" event={"ID":"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2","Type":"ContainerStarted","Data":"89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78"}
Jul 15 12:46:52.965298 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.965139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k" event={"ID":"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2","Type":"ContainerStarted","Data":"b0df652896a857f31335d6a595699cf456cdfd179ba27c33925750cf24b29fa3"}
Jul 15 12:46:52.965361 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.965330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:46:52.965361 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.965354    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:52.966422 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.966400    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:46:52.987068 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:52.987021    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k" podStartSLOduration=1.618762744 podStartE2EDuration="1.987007121s" totalImagesPullingTime="368.244377ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:46:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:46:52.984395871 +0000 UTC m=+1601.861118876" watchObservedRunningTime="2026-07-15 12:46:52.987007121 +0000 UTC m=+1601.863730127"
Jul 15 12:46:58.975369 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:46:58.975341    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:47:01.006289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:01.006204    2576 generic.go:356] "Generic (PLEG): container finished" podID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerID="5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f" exitCode=0
Jul 15 12:47:01.006289 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:01.006249    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" event={"ID":"e863f53a-e258-4d23-bfed-75312a6f90c4","Type":"ContainerDied","Data":"5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"}
Jul 15 12:47:01.006686 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:01.006633    2576 scope.go:117] "RemoveContainer" containerID="5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"
Jul 15 12:47:01.007576 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:01.007548    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:47:01.007672 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:01.007611    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:47:01.007672 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:01.007640    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:47:02.012968 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:02.012935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" event={"ID":"e863f53a-e258-4d23-bfed-75312a6f90c4","Type":"ContainerStarted","Data":"c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9"}
Jul 15 12:47:02.642813 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:02.642779    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:47:02.643159 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:02.643133    2576 patch_prober.go:28] interesting pod/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 15 12:47:02.643326 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:02.643300    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 15 12:47:03.643332 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:03.643227    2576 patch_prober.go:28] interesting pod/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 15 12:47:03.643812 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:03.643317    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 15 12:47:04.642282 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:04.642238    2576 patch_prober.go:28] interesting pod/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 15 12:47:04.642487 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:04.642319    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 15 12:47:05.642435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:05.642384    2576 patch_prober.go:28] interesting pod/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 15 12:47:05.642839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:05.642489    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 15 12:47:06.646120 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:06.646091    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:47:06.646550 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:06.646364    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:47:06.647012 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:06.646995    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:47:06.773227 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:06.773195    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"]
Jul 15 12:47:06.773507 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:06.773449    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k" podUID="2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" containerName="maas-api" containerID="cri-o://89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78" gracePeriod=30
Jul 15 12:47:07.023749 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.023723    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:47:07.040101 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.040074    2576 generic.go:356] "Generic (PLEG): container finished" podID="2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" containerID="89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78" exitCode=0
Jul 15 12:47:07.040237 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.040110    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"
Jul 15 12:47:07.040237 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.040208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k" event={"ID":"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2","Type":"ContainerDied","Data":"89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78"}
Jul 15 12:47:07.040355 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.040273    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k" event={"ID":"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2","Type":"ContainerDied","Data":"b0df652896a857f31335d6a595699cf456cdfd179ba27c33925750cf24b29fa3"}
Jul 15 12:47:07.040355 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.040308    2576 scope.go:117] "RemoveContainer" containerID="89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78"
Jul 15 12:47:07.050555 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.050518    2576 scope.go:117] "RemoveContainer" containerID="89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78"
Jul 15 12:47:07.050834 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:47:07.050814    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78\": container with ID starting with 89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78 not found: ID does not exist" containerID="89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78"
Jul 15 12:47:07.050917 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.050841    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78"} err="failed to get container status \"89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78\": rpc error: code = NotFound desc = could not find container \"89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78\": container with ID starting with 89b0ad2d9c401fccab28bb7068cc0e868116243ace18c64506c564eec4ce2d78 not found: ID does not exist"
Jul 15 12:47:07.181523 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.181496    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-maas-api-tls\") pod \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") "
Jul 15 12:47:07.181708 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.181624    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-openshift-service-ca\") pod \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") "
Jul 15 12:47:07.181708 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.181662    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98q87\" (UniqueName: \"kubernetes.io/projected/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-kube-api-access-98q87\") pod \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\" (UID: \"2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2\") "
Jul 15 12:47:07.182012 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.181983    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" (UID: "2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:47:07.183430 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.183403    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" (UID: "2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:47:07.183575 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.183519    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-kube-api-access-98q87" (OuterVolumeSpecName: "kube-api-access-98q87") pod "2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" (UID: "2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2"). InnerVolumeSpecName "kube-api-access-98q87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:47:07.282723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.282672    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:07.282723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.282692    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98q87\" (UniqueName: \"kubernetes.io/projected/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-kube-api-access-98q87\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:07.282723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.282703    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:07.363431 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.363409    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"]
Jul 15 12:47:07.369379 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.369354    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-907cfafc-5bd5d7b56c-srm5k"]
Jul 15 12:47:07.572497 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:07.572428    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" path="/var/lib/kubelet/pods/2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2/volumes"
Jul 15 12:47:21.495473 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:21.495440    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"]
Jul 15 12:47:21.495964 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:21.495666    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" containerID="cri-o://c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9" gracePeriod=30
Jul 15 12:47:21.648253 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:21.648219    2576 patch_prober.go:28] interesting pod/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 15 12:47:21.648426 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:21.648294    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 12:47:23.264541 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.264492    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"]
Jul 15 12:47:23.265050 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.265035    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" containerName="maas-api"
Jul 15 12:47:23.265091 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.265052    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" containerName="maas-api"
Jul 15 12:47:23.265162 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.265152    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fd3fb95-bff1-4efd-a9e8-aeed0517b4f2" containerName="maas-api"
Jul 15 12:47:23.268523 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.268504    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.271278 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.271260    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-691083d1-serving-cert\""
Jul 15 12:47:23.286694 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.286670    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"]
Jul 15 12:47:23.301754 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.301724    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f090b0d0-0cab-44ae-8fb5-3e841ec43535-openshift-service-ca\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.301859 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.301766    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.301859 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.301793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq6xp\" (UniqueName: \"kubernetes.io/projected/f090b0d0-0cab-44ae-8fb5-3e841ec43535-kube-api-access-fq6xp\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.402911 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.402878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f090b0d0-0cab-44ae-8fb5-3e841ec43535-openshift-service-ca\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.402911 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.402914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.403116 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.402932    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq6xp\" (UniqueName: \"kubernetes.io/projected/f090b0d0-0cab-44ae-8fb5-3e841ec43535-kube-api-access-fq6xp\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.403116 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:47:23.403049    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-serving-cert: secret "maas-api-e2e-ait-691083d1-serving-cert" not found
Jul 15 12:47:23.403206 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:47:23.403139    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls podName:f090b0d0-0cab-44ae-8fb5-3e841ec43535 nodeName:}" failed. No retries permitted until 2026-07-15 12:47:23.903119625 +0000 UTC m=+1632.779842617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls") pod "maas-api-e2e-ait-691083d1-6848775769-b992p" (UID: "f090b0d0-0cab-44ae-8fb5-3e841ec43535") : secret "maas-api-e2e-ait-691083d1-serving-cert" not found
Jul 15 12:47:23.403506 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.403488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f090b0d0-0cab-44ae-8fb5-3e841ec43535-openshift-service-ca\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.422602 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.422573    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq6xp\" (UniqueName: \"kubernetes.io/projected/f090b0d0-0cab-44ae-8fb5-3e841ec43535-kube-api-access-fq6xp\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.907336 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.907301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:23.909660 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:23.909639    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls\") pod \"maas-api-e2e-ait-691083d1-6848775769-b992p\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:24.179339 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:24.179264    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:24.336652 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:24.336626    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"]
Jul 15 12:47:24.337033 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:47:24.337011    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf090b0d0_0cab_44ae_8fb5_3e841ec43535.slice/crio-1ab7fbe6f1c0b7faa4747c4809448a1b7cc243ef1d6f3ee58c37e049f0bb3dc6 WatchSource:0}: Error finding container 1ab7fbe6f1c0b7faa4747c4809448a1b7cc243ef1d6f3ee58c37e049f0bb3dc6: Status 404 returned error can't find the container with id 1ab7fbe6f1c0b7faa4747c4809448a1b7cc243ef1d6f3ee58c37e049f0bb3dc6
Jul 15 12:47:25.138281 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:25.138193    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p" event={"ID":"f090b0d0-0cab-44ae-8fb5-3e841ec43535","Type":"ContainerStarted","Data":"7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2"}
Jul 15 12:47:25.138281 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:25.138242    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p" event={"ID":"f090b0d0-0cab-44ae-8fb5-3e841ec43535","Type":"ContainerStarted","Data":"1ab7fbe6f1c0b7faa4747c4809448a1b7cc243ef1d6f3ee58c37e049f0bb3dc6"}
Jul 15 12:47:25.138469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:25.138297    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:47:25.199376 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:25.199321    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p" podStartSLOduration=1.770328197 podStartE2EDuration="2.199306447s" totalImagesPullingTime="428.97825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:47:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:47:25.196737837 +0000 UTC m=+1634.073460842" watchObservedRunningTime="2026-07-15 12:47:25.199306447 +0000 UTC m=+1634.076029452"
Jul 15 12:47:26.753593 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.753567    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:47:26.831199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831171    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-token\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831345 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831257    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-socket\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831345 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831285    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-data\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831345 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831307    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-podinfo\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831500 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831359    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-certs\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831500 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831395    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-envoy\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831500 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831436    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5pck\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-kube-api-access-z5pck\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831500 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831471    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e863f53a-e258-4d23-bfed-75312a6f90c4-istiod-ca-cert\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831692 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831521    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-credential-socket\") pod \"e863f53a-e258-4d23-bfed-75312a6f90c4\" (UID: \"e863f53a-e258-4d23-bfed-75312a6f90c4\") "
Jul 15 12:47:26.831692 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831602    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:47:26.831870 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.831843    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.832057 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.832017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:47:26.832140 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.832054    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:47:26.832695 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.832658    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e863f53a-e258-4d23-bfed-75312a6f90c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:47:26.832837 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.832797    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:47:26.833821 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.833796    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 12:47:26.834034 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.834011    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:47:26.834160 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.834142    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:47:26.834366 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.834349    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-kube-api-access-z5pck" (OuterVolumeSpecName: "kube-api-access-z5pck") pod "e863f53a-e258-4d23-bfed-75312a6f90c4" (UID: "e863f53a-e258-4d23-bfed-75312a6f90c4"). InnerVolumeSpecName "kube-api-access-z5pck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:47:26.933208 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933178    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-credential-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933208 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933204    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933208 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933214    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-data\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933222    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-podinfo\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933229    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-workload-certs\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933236    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e863f53a-e258-4d23-bfed-75312a6f90c4-istio-envoy\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933245    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5pck\" (UniqueName: \"kubernetes.io/projected/e863f53a-e258-4d23-bfed-75312a6f90c4-kube-api-access-z5pck\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:26.933402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:26.933253    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e863f53a-e258-4d23-bfed-75312a6f90c4-istiod-ca-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:47:27.150155 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.150117    2576 generic.go:356] "Generic (PLEG): container finished" podID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerID="c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9" exitCode=0
Jul 15 12:47:27.150300 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.150173    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"
Jul 15 12:47:27.150300 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.150170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" event={"ID":"e863f53a-e258-4d23-bfed-75312a6f90c4","Type":"ContainerDied","Data":"c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9"}
Jul 15 12:47:27.150300 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.150282    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t" event={"ID":"e863f53a-e258-4d23-bfed-75312a6f90c4","Type":"ContainerDied","Data":"cd7719bc726b25690d533585e9d2f200cda194d73950c97f09910b63a6fa0cf6"}
Jul 15 12:47:27.150411 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.150304    2576 scope.go:117] "RemoveContainer" containerID="c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9"
Jul 15 12:47:27.159426 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.159410    2576 scope.go:117] "RemoveContainer" containerID="5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"
Jul 15 12:47:27.166930 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.166914    2576 scope.go:117] "RemoveContainer" containerID="c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9"
Jul 15 12:47:27.167159 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:47:27.167142    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9\": container with ID starting with c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9 not found: ID does not exist" containerID="c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9"
Jul 15 12:47:27.167202 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.167165    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9"} err="failed to get container status \"c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9\": rpc error: code = NotFound desc = could not find container \"c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9\": container with ID starting with c923e309184b74b806f795446d2c654688197cbb9978795bf4150d7b6d114ae9 not found: ID does not exist"
Jul 15 12:47:27.167202 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.167180    2576 scope.go:117] "RemoveContainer" containerID="5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"
Jul 15 12:47:27.167382 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:47:27.167364    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f\": container with ID starting with 5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f not found: ID does not exist" containerID="5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"
Jul 15 12:47:27.167436 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.167391    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f"} err="failed to get container status \"5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f\": rpc error: code = NotFound desc = could not find container \"5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f\": container with ID starting with 5de56bcace27baab01f9b31c55ca02036a09b9dea608e9e624bada59df14b26f not found: ID does not exist"
Jul 15 12:47:27.174982 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.174952    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"]
Jul 15 12:47:27.177158 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.177139    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-907cfafc-gw-openshift-default-6888fbbf4c-f7z9t"]
Jul 15 12:47:27.573095 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:27.573020    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" path="/var/lib/kubelet/pods/e863f53a-e258-4d23-bfed-75312a6f90c4/volumes"
Jul 15 12:47:31.149129 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:47:31.149102    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:48:28.925949 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:28.925919    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"]
Jul 15 12:48:28.926374 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:28.926130    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p" podUID="f090b0d0-0cab-44ae-8fb5-3e841ec43535" containerName="maas-api" containerID="cri-o://7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2" gracePeriod=30
Jul 15 12:48:29.171820 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.171791    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:48:29.239818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.239747    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f090b0d0-0cab-44ae-8fb5-3e841ec43535-openshift-service-ca\") pod \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") "
Jul 15 12:48:29.239818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.239795    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq6xp\" (UniqueName: \"kubernetes.io/projected/f090b0d0-0cab-44ae-8fb5-3e841ec43535-kube-api-access-fq6xp\") pod \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") "
Jul 15 12:48:29.239970 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.239826    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls\") pod \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\" (UID: \"f090b0d0-0cab-44ae-8fb5-3e841ec43535\") "
Jul 15 12:48:29.240078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.240056    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f090b0d0-0cab-44ae-8fb5-3e841ec43535-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f090b0d0-0cab-44ae-8fb5-3e841ec43535" (UID: "f090b0d0-0cab-44ae-8fb5-3e841ec43535"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:48:29.241868 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.241848    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f090b0d0-0cab-44ae-8fb5-3e841ec43535" (UID: "f090b0d0-0cab-44ae-8fb5-3e841ec43535"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:48:29.241938 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.241909    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f090b0d0-0cab-44ae-8fb5-3e841ec43535-kube-api-access-fq6xp" (OuterVolumeSpecName: "kube-api-access-fq6xp") pod "f090b0d0-0cab-44ae-8fb5-3e841ec43535" (UID: "f090b0d0-0cab-44ae-8fb5-3e841ec43535"). InnerVolumeSpecName "kube-api-access-fq6xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:48:29.341341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.341313    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f090b0d0-0cab-44ae-8fb5-3e841ec43535-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:48:29.341341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.341338    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq6xp\" (UniqueName: \"kubernetes.io/projected/f090b0d0-0cab-44ae-8fb5-3e841ec43535-kube-api-access-fq6xp\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:48:29.341476 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.341347    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f090b0d0-0cab-44ae-8fb5-3e841ec43535-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:48:29.432121 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.432094    2576 generic.go:356] "Generic (PLEG): container finished" podID="f090b0d0-0cab-44ae-8fb5-3e841ec43535" containerID="7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2" exitCode=0
Jul 15 12:48:29.432222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.432137    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"
Jul 15 12:48:29.432222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.432179    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p" event={"ID":"f090b0d0-0cab-44ae-8fb5-3e841ec43535","Type":"ContainerDied","Data":"7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2"}
Jul 15 12:48:29.432290 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.432233    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p" event={"ID":"f090b0d0-0cab-44ae-8fb5-3e841ec43535","Type":"ContainerDied","Data":"1ab7fbe6f1c0b7faa4747c4809448a1b7cc243ef1d6f3ee58c37e049f0bb3dc6"}
Jul 15 12:48:29.432290 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.432258    2576 scope.go:117] "RemoveContainer" containerID="7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2"
Jul 15 12:48:29.441468 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.441450    2576 scope.go:117] "RemoveContainer" containerID="7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2"
Jul 15 12:48:29.441723 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:48:29.441704    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2\": container with ID starting with 7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2 not found: ID does not exist" containerID="7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2"
Jul 15 12:48:29.441809 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.441731    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2"} err="failed to get container status \"7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2\": rpc error: code = NotFound desc = could not find container \"7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2\": container with ID starting with 7ad23dca89a0a7195d2350530287447d41b9e86a7263316656b5bb582c136be2 not found: ID does not exist"
Jul 15 12:48:29.464192 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.464170    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"]
Jul 15 12:48:29.466737 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.466718    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-691083d1-6848775769-b992p"]
Jul 15 12:48:29.571737 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:29.571670    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f090b0d0-0cab-44ae-8fb5-3e841ec43535" path="/var/lib/kubelet/pods/f090b0d0-0cab-44ae-8fb5-3e841ec43535/volumes"
Jul 15 12:48:44.845199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"]
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845495    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845507    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845519    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f090b0d0-0cab-44ae-8fb5-3e841ec43535" containerName="maas-api"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845525    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f090b0d0-0cab-44ae-8fb5-3e841ec43535" containerName="maas-api"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845552    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845559    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845630    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f090b0d0-0cab-44ae-8fb5-3e841ec43535" containerName="maas-api"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845642    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy"
Jul 15 12:48:44.845769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.845648    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="e863f53a-e258-4d23-bfed-75312a6f90c4" containerName="istio-proxy"
Jul 15 12:48:44.848704 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.848683    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.851280 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.851258    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-a074e0bb-openshift-default-dockercfg-g8428\""
Jul 15 12:48:44.864556 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.864507    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"]
Jul 15 12:48:44.965183 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-token\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965326 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965212    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgdrz\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-kube-api-access-tgdrz\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965326 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-envoy\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965401 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965366    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-certs\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-socket\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965430    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-podinfo\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965498 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-credential-socket\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965498 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965488    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-data\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:44.965604 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:44.965546    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istiod-ca-cert\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-socket\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066649    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-podinfo\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066673    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-credential-socket\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-data\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066718    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istiod-ca-cert\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-token\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgdrz\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-kube-api-access-tgdrz\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-envoy\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.066999 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.066900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-certs\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.067096 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.067029    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-socket\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.067142 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.067086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-data\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.067142 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.067113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-credential-socket\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.067260 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.067186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-certs\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.067676 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.067655    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istiod-ca-cert\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.069077 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.069057    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-envoy\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.069302 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.069283    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-podinfo\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.076932 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.076910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-token\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.077394 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.077373    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgdrz\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-kube-api-access-tgdrz\") pod \"e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") " pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.159480 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.159457    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:45.298467 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.298439    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"]
Jul 15 12:48:45.300135 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:48:45.300107    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f2f172a_e85b_407c_b83e_f7faf83c2e75.slice/crio-1f18f3d4a284f1af6a894834ad1b6d6a0bea8f701031819ae74b7669f0ad4216 WatchSource:0}: Error finding container 1f18f3d4a284f1af6a894834ad1b6d6a0bea8f701031819ae74b7669f0ad4216: Status 404 returned error can't find the container with id 1f18f3d4a284f1af6a894834ad1b6d6a0bea8f701031819ae74b7669f0ad4216
Jul 15 12:48:45.302250 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.302222    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:48:45.302327 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.302282    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:48:45.302327 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.302310    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:48:45.515973 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.515884    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" event={"ID":"7f2f172a-e85b-407c-b83e-f7faf83c2e75","Type":"ContainerStarted","Data":"57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"}
Jul 15 12:48:45.516109 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.515981    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" event={"ID":"7f2f172a-e85b-407c-b83e-f7faf83c2e75","Type":"ContainerStarted","Data":"1f18f3d4a284f1af6a894834ad1b6d6a0bea8f701031819ae74b7669f0ad4216"}
Jul 15 12:48:45.538149 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:45.538104    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podStartSLOduration=1.538091119 podStartE2EDuration="1.538091119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:48:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:48:45.536771822 +0000 UTC m=+1714.413494838" watchObservedRunningTime="2026-07-15 12:48:45.538091119 +0000 UTC m=+1714.414814125"
Jul 15 12:48:46.113276 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.113241    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"]
Jul 15 12:48:46.117286 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.117262    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.120609 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.120588    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a074e0bb-serving-cert\""
Jul 15 12:48:46.129814 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.129787    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"]
Jul 15 12:48:46.160124 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.160097    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:46.164771 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.164743    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:46.177028 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.177003    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f844515-2481-4505-9c31-84a121ac388f-openshift-service-ca\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.177169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.177147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkjc2\" (UniqueName: \"kubernetes.io/projected/1f844515-2481-4505-9c31-84a121ac388f-kube-api-access-rkjc2\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.177271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.177180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.277830 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.277799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkjc2\" (UniqueName: \"kubernetes.io/projected/1f844515-2481-4505-9c31-84a121ac388f-kube-api-access-rkjc2\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.277830 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.277835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.278018 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.277869    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f844515-2481-4505-9c31-84a121ac388f-openshift-service-ca\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.278018 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:48:46.277957    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-serving-cert: secret "maas-api-e2e-derive-a074e0bb-serving-cert" not found
Jul 15 12:48:46.278018 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:48:46.278017    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls podName:1f844515-2481-4505-9c31-84a121ac388f nodeName:}" failed. No retries permitted until 2026-07-15 12:48:46.777999986 +0000 UTC m=+1715.654722970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls") pod "maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" (UID: "1f844515-2481-4505-9c31-84a121ac388f") : secret "maas-api-e2e-derive-a074e0bb-serving-cert" not found
Jul 15 12:48:46.278490 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.278473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f844515-2481-4505-9c31-84a121ac388f-openshift-service-ca\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.315465 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.315435    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkjc2\" (UniqueName: \"kubernetes.io/projected/1f844515-2481-4505-9c31-84a121ac388f-kube-api-access-rkjc2\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.520314 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.520279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:46.521059 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.521039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:46.782561 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.782467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:46.784860 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:46.784841    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls\") pod \"maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:47.032078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:47.032047    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:47.193159 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:47.193133    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"]
Jul 15 12:48:47.194679 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:48:47.194652    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f844515_2481_4505_9c31_84a121ac388f.slice/crio-3c23d5b73cd4f787beae1b2c749e8c7ae700cab4caa368b8b7d446b3e5b81d72 WatchSource:0}: Error finding container 3c23d5b73cd4f787beae1b2c749e8c7ae700cab4caa368b8b7d446b3e5b81d72: Status 404 returned error can't find the container with id 3c23d5b73cd4f787beae1b2c749e8c7ae700cab4caa368b8b7d446b3e5b81d72
Jul 15 12:48:47.525719 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:47.525687    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" event={"ID":"1f844515-2481-4505-9c31-84a121ac388f","Type":"ContainerStarted","Data":"3c23d5b73cd4f787beae1b2c749e8c7ae700cab4caa368b8b7d446b3e5b81d72"}
Jul 15 12:48:48.531945 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:48.531910    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" event={"ID":"1f844515-2481-4505-9c31-84a121ac388f","Type":"ContainerStarted","Data":"4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2"}
Jul 15 12:48:48.532371 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:48.532006    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:48.562571 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:48.562509    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" podStartSLOduration=2.121317601 podStartE2EDuration="2.562496137s" totalImagesPullingTime="441.178536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:48:48.559239092 +0000 UTC m=+1717.435962099" watchObservedRunningTime="2026-07-15 12:48:48.562496137 +0000 UTC m=+1717.439219182"
Jul 15 12:48:54.546363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:54.546329    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:48:56.589491 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:56.589458    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerID="57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f" exitCode=0
Jul 15 12:48:56.590035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:56.589548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" event={"ID":"7f2f172a-e85b-407c-b83e-f7faf83c2e75","Type":"ContainerDied","Data":"57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"}
Jul 15 12:48:56.590035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:56.589891    2576 scope.go:117] "RemoveContainer" containerID="57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"
Jul 15 12:48:56.590835 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:56.590811    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:48:56.590895 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:56.590882    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:48:56.590930 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:56.590914    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:48:57.595813 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:57.595784    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" event={"ID":"7f2f172a-e85b-407c-b83e-f7faf83c2e75","Type":"ContainerStarted","Data":"eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a"}
Jul 15 12:48:58.159798 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:58.159766    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:48:58.160996 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:58.160970    2576 patch_prober.go:28] interesting pod/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 15 12:48:58.161122 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:58.161047    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 15 12:48:59.160035 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:59.160000    2576 patch_prober.go:28] interesting pod/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 15 12:48:59.160402 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:48:59.160070    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 15 12:49:00.160761 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:00.160727    2576 patch_prober.go:28] interesting pod/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 15 12:49:00.161141 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:00.160778    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 15 12:49:01.160525 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:01.160487    2576 patch_prober.go:28] interesting pod/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 15 12:49:01.160720 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:01.160566    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 15 12:49:01.520595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:01.520495    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:49:02.164108 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:02.164082    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:49:02.165049 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:49:02.165033    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:50:01.229057 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.229021    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"]
Jul 15 12:50:01.229671 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.229262    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" podUID="1f844515-2481-4505-9c31-84a121ac388f" containerName="maas-api" containerID="cri-o://4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2" gracePeriod=30
Jul 15 12:50:01.476032 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.476009    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:50:01.635078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.634997    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls\") pod \"1f844515-2481-4505-9c31-84a121ac388f\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") "
Jul 15 12:50:01.635078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.635063    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f844515-2481-4505-9c31-84a121ac388f-openshift-service-ca\") pod \"1f844515-2481-4505-9c31-84a121ac388f\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") "
Jul 15 12:50:01.635305 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.635158    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkjc2\" (UniqueName: \"kubernetes.io/projected/1f844515-2481-4505-9c31-84a121ac388f-kube-api-access-rkjc2\") pod \"1f844515-2481-4505-9c31-84a121ac388f\" (UID: \"1f844515-2481-4505-9c31-84a121ac388f\") "
Jul 15 12:50:01.635403 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.635383    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f844515-2481-4505-9c31-84a121ac388f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1f844515-2481-4505-9c31-84a121ac388f" (UID: "1f844515-2481-4505-9c31-84a121ac388f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:50:01.636984 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.636964    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1f844515-2481-4505-9c31-84a121ac388f" (UID: "1f844515-2481-4505-9c31-84a121ac388f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:50:01.637041 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.636996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f844515-2481-4505-9c31-84a121ac388f-kube-api-access-rkjc2" (OuterVolumeSpecName: "kube-api-access-rkjc2") pod "1f844515-2481-4505-9c31-84a121ac388f" (UID: "1f844515-2481-4505-9c31-84a121ac388f"). InnerVolumeSpecName "kube-api-access-rkjc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:50:01.736280 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.736255    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1f844515-2481-4505-9c31-84a121ac388f-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:01.736416 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.736293    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f844515-2481-4505-9c31-84a121ac388f-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:01.736416 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.736318    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkjc2\" (UniqueName: \"kubernetes.io/projected/1f844515-2481-4505-9c31-84a121ac388f-kube-api-access-rkjc2\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:01.899603 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.899570    2576 generic.go:356] "Generic (PLEG): container finished" podID="1f844515-2481-4505-9c31-84a121ac388f" containerID="4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2" exitCode=0
Jul 15 12:50:01.899769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.899636    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"
Jul 15 12:50:01.899769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.899637    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" event={"ID":"1f844515-2481-4505-9c31-84a121ac388f","Type":"ContainerDied","Data":"4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2"}
Jul 15 12:50:01.899769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.899750    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt" event={"ID":"1f844515-2481-4505-9c31-84a121ac388f","Type":"ContainerDied","Data":"3c23d5b73cd4f787beae1b2c749e8c7ae700cab4caa368b8b7d446b3e5b81d72"}
Jul 15 12:50:01.899769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.899770    2576 scope.go:117] "RemoveContainer" containerID="4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2"
Jul 15 12:50:01.908647 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.908630    2576 scope.go:117] "RemoveContainer" containerID="4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2"
Jul 15 12:50:01.908893 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:50:01.908870    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2\": container with ID starting with 4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2 not found: ID does not exist" containerID="4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2"
Jul 15 12:50:01.908934 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.908907    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2"} err="failed to get container status \"4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2\": rpc error: code = NotFound desc = could not find container \"4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2\": container with ID starting with 4452a4601b28038bd8b26e9c16f1f801b17a33c735ba9b86a5eea0196e2898a2 not found: ID does not exist"
Jul 15 12:50:01.926801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.926777    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"]
Jul 15 12:50:01.929618 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:01.929599    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a074e0bb-5794b75f6c-pwxzt"]
Jul 15 12:50:03.573241 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:03.573167    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f844515-2481-4505-9c31-84a121ac388f" path="/var/lib/kubelet/pods/1f844515-2481-4505-9c31-84a121ac388f/volumes"
Jul 15 12:50:11.589242 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:11.589212    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:50:11.593382 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:11.593361    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:50:16.027445 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:16.027411    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"]
Jul 15 12:50:16.027839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:16.027625    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy" containerID="cri-o://eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a" gracePeriod=30
Jul 15 12:50:17.166372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:17.166330    2576 patch_prober.go:28] interesting pod/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 12:50:17.166748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:17.166400    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 12:50:21.279361 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.279338    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:50:21.285415 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285397    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-podinfo\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285489 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285444    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istiod-ca-cert\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285486    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgdrz\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-kube-api-access-tgdrz\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285506    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-data\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285525    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-socket\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285707 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285569    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-envoy\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285707 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-certs\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285707 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285632    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-credential-socket\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285707 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-token\") pod \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\" (UID: \"7f2f172a-e85b-407c-b83e-f7faf83c2e75\") "
Jul 15 12:50:21.285922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285843    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:50:21.285993 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.285976    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.286156 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.286077    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:50:21.286278 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.286192    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:50:21.286278 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.286258    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:50:21.286482 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.286458    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-data" (OuterVolumeSpecName: "istio-data") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:50:21.287557 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.287519    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 12:50:21.287691 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.287583    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:50:21.287963 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.287946    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-kube-api-access-tgdrz" (OuterVolumeSpecName: "kube-api-access-tgdrz") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "kube-api-access-tgdrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:50:21.288031 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.287968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-token" (OuterVolumeSpecName: "istio-token") pod "7f2f172a-e85b-407c-b83e-f7faf83c2e75" (UID: "7f2f172a-e85b-407c-b83e-f7faf83c2e75"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:50:21.387114 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387082    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-podinfo\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387122    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istiod-ca-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387136    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgdrz\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-kube-api-access-tgdrz\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387150    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-data\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387163    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-envoy\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387174    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-workload-certs\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387186    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f2f172a-e85b-407c-b83e-f7faf83c2e75-credential-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.387234 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.387198    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f2f172a-e85b-407c-b83e-f7faf83c2e75-istio-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:50:21.995910 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.995874    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerID="eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a" exitCode=0
Jul 15 12:50:21.996094 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.995937    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"
Jul 15 12:50:21.996094 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.995946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" event={"ID":"7f2f172a-e85b-407c-b83e-f7faf83c2e75","Type":"ContainerDied","Data":"eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a"}
Jul 15 12:50:21.996094 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.995988    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7" event={"ID":"7f2f172a-e85b-407c-b83e-f7faf83c2e75","Type":"ContainerDied","Data":"1f18f3d4a284f1af6a894834ad1b6d6a0bea8f701031819ae74b7669f0ad4216"}
Jul 15 12:50:21.996094 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:21.996009    2576 scope.go:117] "RemoveContainer" containerID="eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a"
Jul 15 12:50:22.004844 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.004828    2576 scope.go:117] "RemoveContainer" containerID="57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"
Jul 15 12:50:22.012436 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.012420    2576 scope.go:117] "RemoveContainer" containerID="eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a"
Jul 15 12:50:22.012707 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:50:22.012687    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a\": container with ID starting with eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a not found: ID does not exist" containerID="eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a"
Jul 15 12:50:22.012772 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.012714    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a"} err="failed to get container status \"eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a\": rpc error: code = NotFound desc = could not find container \"eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a\": container with ID starting with eeb0ce880dab3bcccb6554f4fd1ddd6347040115d9f7ea07b118a9899d70693a not found: ID does not exist"
Jul 15 12:50:22.012772 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.012731    2576 scope.go:117] "RemoveContainer" containerID="57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"
Jul 15 12:50:22.012925 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:50:22.012908    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f\": container with ID starting with 57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f not found: ID does not exist" containerID="57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"
Jul 15 12:50:22.012974 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.012932    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f"} err="failed to get container status \"57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f\": rpc error: code = NotFound desc = could not find container \"57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f\": container with ID starting with 57898e966d496664c0c6e818d56853e2de4fa3811c0993233ab6ca1d61e21c7f not found: ID does not exist"
Jul 15 12:50:22.029670 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.029649    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"]
Jul 15 12:50:22.040704 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:22.040686    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-a074e0bb-openshift-default-c9d9b57c9-dj5b7"]
Jul 15 12:50:23.572326 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:50:23.572292    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" path="/var/lib/kubelet/pods/7f2f172a-e85b-407c-b83e-f7faf83c2e75/volumes"
Jul 15 12:52:00.716224 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716141    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"]
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716496    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f844515-2481-4505-9c31-84a121ac388f" containerName="maas-api"
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716507    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f844515-2481-4505-9c31-84a121ac388f" containerName="maas-api"
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716517    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy"
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716524    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy"
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716621    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy"
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716630    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f844515-2481-4505-9c31-84a121ac388f" containerName="maas-api"
Jul 15 12:52:00.716793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.716639    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy"
Jul 15 12:52:00.719683 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.719665    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.723078 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.723060    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8a28be15-serving-cert\""
Jul 15 12:52:00.730745 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.730722    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"]
Jul 15 12:52:00.781517 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.781481    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25933ee6-9549-481b-b357-969cadc5fc68-openshift-service-ca\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.781517 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.781514    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.781723 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.781617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd2qv\" (UniqueName: \"kubernetes.io/projected/25933ee6-9549-481b-b357-969cadc5fc68-kube-api-access-nd2qv\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.882222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.882191    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25933ee6-9549-481b-b357-969cadc5fc68-openshift-service-ca\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.882222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.882222    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.882457 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.882261    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd2qv\" (UniqueName: \"kubernetes.io/projected/25933ee6-9549-481b-b357-969cadc5fc68-kube-api-access-nd2qv\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.882457 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:52:00.882368    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-serving-cert: secret "maas-api-e2e-mt-8a28be15-serving-cert" not found
Jul 15 12:52:00.882604 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:52:00.882460    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls podName:25933ee6-9549-481b-b357-969cadc5fc68 nodeName:}" failed. No retries permitted until 2026-07-15 12:52:01.382437569 +0000 UTC m=+1910.259160576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls") pod "maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" (UID: "25933ee6-9549-481b-b357-969cadc5fc68") : secret "maas-api-e2e-mt-8a28be15-serving-cert" not found
Jul 15 12:52:00.882827 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.882808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25933ee6-9549-481b-b357-969cadc5fc68-openshift-service-ca\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:00.896380 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:00.896353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd2qv\" (UniqueName: \"kubernetes.io/projected/25933ee6-9549-481b-b357-969cadc5fc68-kube-api-access-nd2qv\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:01.386793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:01.386760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:01.389045 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:01.389023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls\") pod \"maas-api-e2e-mt-8a28be15-7d55586d96-crkz9\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:01.631469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:01.631433    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:01.977741 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:01.977712    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"]
Jul 15 12:52:01.979421 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:52:01.979383    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25933ee6_9549_481b_b357_969cadc5fc68.slice/crio-96fc5f30ead6ff573ab413aef7653761ab0ce44ed4b18486d922995924f2ef20 WatchSource:0}: Error finding container 96fc5f30ead6ff573ab413aef7653761ab0ce44ed4b18486d922995924f2ef20: Status 404 returned error can't find the container with id 96fc5f30ead6ff573ab413aef7653761ab0ce44ed4b18486d922995924f2ef20
Jul 15 12:52:01.980869 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:01.980848    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:52:02.479054 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:02.479019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" event={"ID":"25933ee6-9549-481b-b357-969cadc5fc68","Type":"ContainerStarted","Data":"96fc5f30ead6ff573ab413aef7653761ab0ce44ed4b18486d922995924f2ef20"}
Jul 15 12:52:03.485597 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:03.485565    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" event={"ID":"25933ee6-9549-481b-b357-969cadc5fc68","Type":"ContainerStarted","Data":"01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4"}
Jul 15 12:52:03.485981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:03.485624    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:03.504833 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:03.504784    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" podStartSLOduration=2.948592808 podStartE2EDuration="3.504765574s" totalImagesPullingTime="556.172766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:52:03.504052536 +0000 UTC m=+1912.380775548" watchObservedRunningTime="2026-07-15 12:52:03.504765574 +0000 UTC m=+1912.381488582"
Jul 15 12:52:09.495985 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:09.495953    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:23.244979 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.244931    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"]
Jul 15 12:52:23.245433 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.245250    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" podUID="25933ee6-9549-481b-b357-969cadc5fc68" containerName="maas-api" containerID="cri-o://01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4" gracePeriod=30
Jul 15 12:52:23.487555 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.487520    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:23.584938 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.584874    2576 generic.go:356] "Generic (PLEG): container finished" podID="25933ee6-9549-481b-b357-969cadc5fc68" containerID="01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4" exitCode=0
Jul 15 12:52:23.584938 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.584916    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"
Jul 15 12:52:23.584938 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.584929    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" event={"ID":"25933ee6-9549-481b-b357-969cadc5fc68","Type":"ContainerDied","Data":"01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4"}
Jul 15 12:52:23.585114 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.584965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9" event={"ID":"25933ee6-9549-481b-b357-969cadc5fc68","Type":"ContainerDied","Data":"96fc5f30ead6ff573ab413aef7653761ab0ce44ed4b18486d922995924f2ef20"}
Jul 15 12:52:23.585114 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.584985    2576 scope.go:117] "RemoveContainer" containerID="01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4"
Jul 15 12:52:23.593066 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.593049    2576 scope.go:117] "RemoveContainer" containerID="01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4"
Jul 15 12:52:23.593323 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:52:23.593305    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4\": container with ID starting with 01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4 not found: ID does not exist" containerID="01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4"
Jul 15 12:52:23.593382 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.593332    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4"} err="failed to get container status \"01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4\": rpc error: code = NotFound desc = could not find container \"01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4\": container with ID starting with 01fe7541e95c811f56e1f0c865b1c6b792c23cc268401a000f0cf3781a955ee4 not found: ID does not exist"
Jul 15 12:52:23.674482 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.674460    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls\") pod \"25933ee6-9549-481b-b357-969cadc5fc68\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") "
Jul 15 12:52:23.674626 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.674498    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25933ee6-9549-481b-b357-969cadc5fc68-openshift-service-ca\") pod \"25933ee6-9549-481b-b357-969cadc5fc68\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") "
Jul 15 12:52:23.674626 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.674525    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nd2qv\" (UniqueName: \"kubernetes.io/projected/25933ee6-9549-481b-b357-969cadc5fc68-kube-api-access-nd2qv\") pod \"25933ee6-9549-481b-b357-969cadc5fc68\" (UID: \"25933ee6-9549-481b-b357-969cadc5fc68\") "
Jul 15 12:52:23.674852 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.674831    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25933ee6-9549-481b-b357-969cadc5fc68-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "25933ee6-9549-481b-b357-969cadc5fc68" (UID: "25933ee6-9549-481b-b357-969cadc5fc68"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:52:23.676514 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.676485    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25933ee6-9549-481b-b357-969cadc5fc68-kube-api-access-nd2qv" (OuterVolumeSpecName: "kube-api-access-nd2qv") pod "25933ee6-9549-481b-b357-969cadc5fc68" (UID: "25933ee6-9549-481b-b357-969cadc5fc68"). InnerVolumeSpecName "kube-api-access-nd2qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:52:23.676620 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.676493    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "25933ee6-9549-481b-b357-969cadc5fc68" (UID: "25933ee6-9549-481b-b357-969cadc5fc68"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:52:23.775579 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.775555    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/25933ee6-9549-481b-b357-969cadc5fc68-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:52:23.775579 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.775578    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/25933ee6-9549-481b-b357-969cadc5fc68-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:52:23.775705 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.775587    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nd2qv\" (UniqueName: \"kubernetes.io/projected/25933ee6-9549-481b-b357-969cadc5fc68-kube-api-access-nd2qv\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:52:23.906103 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.906080    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"]
Jul 15 12:52:23.910054 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:23.910034    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8a28be15-7d55586d96-crkz9"]
Jul 15 12:52:25.572362 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:25.572332    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25933ee6-9549-481b-b357-969cadc5fc68" path="/var/lib/kubelet/pods/25933ee6-9549-481b-b357-969cadc5fc68/volumes"
Jul 15 12:52:59.208055 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.208023    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"]
Jul 15 12:52:59.208587 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.208365    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25933ee6-9549-481b-b357-969cadc5fc68" containerName="maas-api"
Jul 15 12:52:59.208587 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.208375    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="25933ee6-9549-481b-b357-969cadc5fc68" containerName="maas-api"
Jul 15 12:52:59.208587 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.208384    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy"
Jul 15 12:52:59.208587 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.208389    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f2f172a-e85b-407c-b83e-f7faf83c2e75" containerName="istio-proxy"
Jul 15 12:52:59.208587 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.208471    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="25933ee6-9549-481b-b357-969cadc5fc68" containerName="maas-api"
Jul 15 12:52:59.211042 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.211023    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.213562 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.213526    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1f25dafc-openshift-default-dockercfg-x9sfk\""
Jul 15 12:52:59.224899 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.224870    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"]
Jul 15 12:52:59.259643 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259618    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-istio-token\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259741 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259647    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9gmc\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-kube-api-access-l9gmc\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259741 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259667    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-credential-socket\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259741 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259683    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202794ac-3921-4e2d-9cdd-be13b7668444-istio-podinfo\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259865 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259758    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-envoy\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259865 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-data\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259865 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259842    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-socket\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259971 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259865    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202794ac-3921-4e2d-9cdd-be13b7668444-istiod-ca-cert\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.259971 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.259889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-certs\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360457 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-socket\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360616 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360467    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202794ac-3921-4e2d-9cdd-be13b7668444-istiod-ca-cert\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360616 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360492    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-certs\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360616 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-istio-token\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360616 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9gmc\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-kube-api-access-l9gmc\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360639    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-credential-socket\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360665    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202794ac-3921-4e2d-9cdd-be13b7668444-istio-podinfo\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-envoy\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.360839 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360725    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-data\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.361034 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.360987    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-certs\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.361034 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.361016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-credential-socket\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.361157 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.361069    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-socket\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.361266 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.361249    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-data\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.361364 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.361287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202794ac-3921-4e2d-9cdd-be13b7668444-istiod-ca-cert\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.363084 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.363065    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-envoy\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.363248 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.363227    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202794ac-3921-4e2d-9cdd-be13b7668444-istio-podinfo\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.368760 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.368743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-istio-token\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.368885 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.368865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9gmc\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-kube-api-access-l9gmc\") pod \"e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") " pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.521027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.520977    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:52:59.660082 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.660052    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"]
Jul 15 12:52:59.664118 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:52:59.664085    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202794ac_3921_4e2d_9cdd_be13b7668444.slice/crio-0b139085a6f4f46df675760678336c98176262501dc680f43fcc9eca82958f99 WatchSource:0}: Error finding container 0b139085a6f4f46df675760678336c98176262501dc680f43fcc9eca82958f99: Status 404 returned error can't find the container with id 0b139085a6f4f46df675760678336c98176262501dc680f43fcc9eca82958f99
Jul 15 12:52:59.666260 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.666228    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:52:59.666350 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.666295    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:52:59.666350 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.666323    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:52:59.762256 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.762198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" event={"ID":"202794ac-3921-4e2d-9cdd-be13b7668444","Type":"ContainerStarted","Data":"6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"}
Jul 15 12:52:59.762394 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.762344    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" event={"ID":"202794ac-3921-4e2d-9cdd-be13b7668444","Type":"ContainerStarted","Data":"0b139085a6f4f46df675760678336c98176262501dc680f43fcc9eca82958f99"}
Jul 15 12:52:59.784756 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:52:59.784669    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podStartSLOduration=0.784654403 podStartE2EDuration="784.654403ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:52:59.783285369 +0000 UTC m=+1968.660008410" watchObservedRunningTime="2026-07-15 12:52:59.784654403 +0000 UTC m=+1968.661377408"
Jul 15 12:53:00.521874 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:00.521836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:00.526446 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:00.526421    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:00.767429 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:00.767404    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:00.768169 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:00.768150    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:01.893030 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.893002    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"]
Jul 15 12:53:01.896780 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.896758    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:01.899856 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.899837    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-1f25dafc-serving-cert\""
Jul 15 12:53:01.915769 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.915749    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"]
Jul 15 12:53:01.987222 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.987197    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/722e869e-f1d2-4cfe-9439-327c11873dbb-openshift-service-ca\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:01.987325 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.987256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/722e869e-f1d2-4cfe-9439-327c11873dbb-maas-api-tls\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:01.987375 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:01.987326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fplg6\" (UniqueName: \"kubernetes.io/projected/722e869e-f1d2-4cfe-9439-327c11873dbb-kube-api-access-fplg6\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.087888 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.087856    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/722e869e-f1d2-4cfe-9439-327c11873dbb-maas-api-tls\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.088047 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.087907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fplg6\" (UniqueName: \"kubernetes.io/projected/722e869e-f1d2-4cfe-9439-327c11873dbb-kube-api-access-fplg6\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.088047 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.087946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/722e869e-f1d2-4cfe-9439-327c11873dbb-openshift-service-ca\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.088581 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.088563    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/722e869e-f1d2-4cfe-9439-327c11873dbb-openshift-service-ca\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.090391 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.090364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/722e869e-f1d2-4cfe-9439-327c11873dbb-maas-api-tls\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.098889 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.098865    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fplg6\" (UniqueName: \"kubernetes.io/projected/722e869e-f1d2-4cfe-9439-327c11873dbb-kube-api-access-fplg6\") pod \"maas-api-e2e-mt-1f25dafc-6669574776-4q4fp\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.207182 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.207157    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:02.345745 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.345716    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"]
Jul 15 12:53:02.348948 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:53:02.348914    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod722e869e_f1d2_4cfe_9439_327c11873dbb.slice/crio-b37ff26496c9b962e2db5a9c6d2cd8ba00b1fc5894baec953929eed95a98a2f4 WatchSource:0}: Error finding container b37ff26496c9b962e2db5a9c6d2cd8ba00b1fc5894baec953929eed95a98a2f4: Status 404 returned error can't find the container with id b37ff26496c9b962e2db5a9c6d2cd8ba00b1fc5894baec953929eed95a98a2f4
Jul 15 12:53:02.776815 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:02.776779    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp" event={"ID":"722e869e-f1d2-4cfe-9439-327c11873dbb","Type":"ContainerStarted","Data":"b37ff26496c9b962e2db5a9c6d2cd8ba00b1fc5894baec953929eed95a98a2f4"}
Jul 15 12:53:03.782629 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:03.782551    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp" event={"ID":"722e869e-f1d2-4cfe-9439-327c11873dbb","Type":"ContainerStarted","Data":"174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431"}
Jul 15 12:53:03.782967 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:03.782687    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:03.810451 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:03.810405    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp" podStartSLOduration=2.381554012 podStartE2EDuration="2.810390922s" totalImagesPullingTime="428.83691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:53:03.808557411 +0000 UTC m=+1972.685280417" watchObservedRunningTime="2026-07-15 12:53:03.810390922 +0000 UTC m=+1972.687113927"
Jul 15 12:53:09.793149 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:09.793114    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:16.614086 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.612850    2576 patch_prober.go:28] interesting pod/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:34536->10.132.0.53:15021: read: connection reset by peer" start-of-body=
Jul 15 12:53:16.614086 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.612919    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": read tcp 10.132.0.2:34536->10.132.0.53:15021: read: connection reset by peer"
Jul 15 12:53:16.643839 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:53:16.643808    2576 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod202794ac_3921_4e2d_9cdd_be13b7668444.slice/crio-conmon-6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0.scope\": RecentStats: unable to find data in memory cache], [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@3-22340-0.service\": RecentStats: unable to find data in memory cache]"
Jul 15 12:53:16.849243 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.849176    2576 generic.go:356] "Generic (PLEG): container finished" podID="202794ac-3921-4e2d-9cdd-be13b7668444" containerID="6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0" exitCode=0
Jul 15 12:53:16.849368 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.849243    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" event={"ID":"202794ac-3921-4e2d-9cdd-be13b7668444","Type":"ContainerDied","Data":"6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"}
Jul 15 12:53:16.849719 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.849694    2576 scope.go:117] "RemoveContainer" containerID="6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"
Jul 15 12:53:16.850728 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.850699    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:53:16.850840 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.850778    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:53:16.850840 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:16.850826    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:53:17.855594 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:17.855557    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" event={"ID":"202794ac-3921-4e2d-9cdd-be13b7668444","Type":"ContainerStarted","Data":"b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae"}
Jul 15 12:53:18.522180 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:18.522150    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:18.522801 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:18.522770    2576 patch_prober.go:28] interesting pod/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 15 12:53:18.523016 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:18.522989    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 15 12:53:19.521555 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:19.521504    2576 patch_prober.go:28] interesting pod/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 15 12:53:19.522037 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:19.521588    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 15 12:53:20.521957 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:20.521921    2576 patch_prober.go:28] interesting pod/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 15 12:53:20.522560 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:20.522490    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 15 12:53:21.521548 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:21.521506    2576 patch_prober.go:28] interesting pod/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 15 12:53:21.521764 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:21.521605    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 15 12:53:22.525323 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:22.525292    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:22.525698 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:22.525562    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:22.526171 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:22.526154    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:53:44.885850 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:44.885791    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"]
Jul 15 12:53:44.886457 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:44.886029    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp" podUID="722e869e-f1d2-4cfe-9439-327c11873dbb" containerName="maas-api" containerID="cri-o://174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431" gracePeriod=30
Jul 15 12:53:45.139551 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.139480    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:45.268271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.268240    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/722e869e-f1d2-4cfe-9439-327c11873dbb-openshift-service-ca\") pod \"722e869e-f1d2-4cfe-9439-327c11873dbb\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") "
Jul 15 12:53:45.268271 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.268272    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fplg6\" (UniqueName: \"kubernetes.io/projected/722e869e-f1d2-4cfe-9439-327c11873dbb-kube-api-access-fplg6\") pod \"722e869e-f1d2-4cfe-9439-327c11873dbb\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") "
Jul 15 12:53:45.268485 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.268296    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/722e869e-f1d2-4cfe-9439-327c11873dbb-maas-api-tls\") pod \"722e869e-f1d2-4cfe-9439-327c11873dbb\" (UID: \"722e869e-f1d2-4cfe-9439-327c11873dbb\") "
Jul 15 12:53:45.268588 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.268566    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/722e869e-f1d2-4cfe-9439-327c11873dbb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "722e869e-f1d2-4cfe-9439-327c11873dbb" (UID: "722e869e-f1d2-4cfe-9439-327c11873dbb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:53:45.270339 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.270313    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/722e869e-f1d2-4cfe-9439-327c11873dbb-kube-api-access-fplg6" (OuterVolumeSpecName: "kube-api-access-fplg6") pod "722e869e-f1d2-4cfe-9439-327c11873dbb" (UID: "722e869e-f1d2-4cfe-9439-327c11873dbb"). InnerVolumeSpecName "kube-api-access-fplg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:53:45.270464 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.270353    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/722e869e-f1d2-4cfe-9439-327c11873dbb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "722e869e-f1d2-4cfe-9439-327c11873dbb" (UID: "722e869e-f1d2-4cfe-9439-327c11873dbb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:53:45.369210 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.369189    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/722e869e-f1d2-4cfe-9439-327c11873dbb-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:53:45.369210 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.369211    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fplg6\" (UniqueName: \"kubernetes.io/projected/722e869e-f1d2-4cfe-9439-327c11873dbb-kube-api-access-fplg6\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:53:45.369210 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.369221    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/722e869e-f1d2-4cfe-9439-327c11873dbb-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:53:45.982884 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.982851    2576 generic.go:356] "Generic (PLEG): container finished" podID="722e869e-f1d2-4cfe-9439-327c11873dbb" containerID="174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431" exitCode=0
Jul 15 12:53:45.983333 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.982899    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"
Jul 15 12:53:45.983333 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.982946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp" event={"ID":"722e869e-f1d2-4cfe-9439-327c11873dbb","Type":"ContainerDied","Data":"174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431"}
Jul 15 12:53:45.983333 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.983018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp" event={"ID":"722e869e-f1d2-4cfe-9439-327c11873dbb","Type":"ContainerDied","Data":"b37ff26496c9b962e2db5a9c6d2cd8ba00b1fc5894baec953929eed95a98a2f4"}
Jul 15 12:53:45.983333 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.983052    2576 scope.go:117] "RemoveContainer" containerID="174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431"
Jul 15 12:53:45.991731 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.991698    2576 scope.go:117] "RemoveContainer" containerID="174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431"
Jul 15 12:53:45.991960 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:53:45.991938    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431\": container with ID starting with 174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431 not found: ID does not exist" containerID="174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431"
Jul 15 12:53:45.992047 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.991966    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431"} err="failed to get container status \"174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431\": rpc error: code = NotFound desc = could not find container \"174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431\": container with ID starting with 174453d543471b278fd714ce933587155aed10c9d0bcb8d1352ec1f5bdae0431 not found: ID does not exist"
Jul 15 12:53:45.998903 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:45.998883    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"]
Jul 15 12:53:46.002764 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:46.002746    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-1f25dafc-6669574776-4q4fp"]
Jul 15 12:53:47.572285 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:53:47.572247    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="722e869e-f1d2-4cfe-9439-327c11873dbb" path="/var/lib/kubelet/pods/722e869e-f1d2-4cfe-9439-327c11873dbb/volumes"
Jul 15 12:54:51.895876 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:51.895795    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"]
Jul 15 12:54:51.896342 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:51.896005    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" containerID="cri-o://b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae" gracePeriod=30
Jul 15 12:54:52.528189 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:52.528156    2576 patch_prober.go:28] interesting pod/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 12:54:52.528340 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:52.528209    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 12:54:57.143407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.143380    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:54:57.267404 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267332    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-data\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267404 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267382    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9gmc\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-kube-api-access-l9gmc\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267614 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267448    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202794ac-3921-4e2d-9cdd-be13b7668444-istiod-ca-cert\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267614 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267488    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-envoy\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267614 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267562    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202794ac-3921-4e2d-9cdd-be13b7668444-istio-podinfo\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267771 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267612    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-credential-socket\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267771 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267639    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-certs\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267771 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267667    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-istio-token\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.267771 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-socket\") pod \"202794ac-3921-4e2d-9cdd-be13b7668444\" (UID: \"202794ac-3921-4e2d-9cdd-be13b7668444\") "
Jul 15 12:54:57.268036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267889    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/202794ac-3921-4e2d-9cdd-be13b7668444-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:54:57.268036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:54:57.268036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.267997    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:54:57.268190 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.268084    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:54:57.268190 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.268143    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/202794ac-3921-4e2d-9cdd-be13b7668444-istiod-ca-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.268297 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.268189    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-credential-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.268297 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.268205    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-certs\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.268297 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.268275    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-data" (OuterVolumeSpecName: "istio-data") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:54:57.269844 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.269813    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:54:57.269939 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.269906    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-kube-api-access-l9gmc" (OuterVolumeSpecName: "kube-api-access-l9gmc") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "kube-api-access-l9gmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:54:57.270116 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.270099    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/202794ac-3921-4e2d-9cdd-be13b7668444-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 12:54:57.270176 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.270124    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-istio-token" (OuterVolumeSpecName: "istio-token") pod "202794ac-3921-4e2d-9cdd-be13b7668444" (UID: "202794ac-3921-4e2d-9cdd-be13b7668444"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:54:57.329070 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.329042    2576 generic.go:356] "Generic (PLEG): container finished" podID="202794ac-3921-4e2d-9cdd-be13b7668444" containerID="b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae" exitCode=0
Jul 15 12:54:57.329185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.329086    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"
Jul 15 12:54:57.329185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.329090    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" event={"ID":"202794ac-3921-4e2d-9cdd-be13b7668444","Type":"ContainerDied","Data":"b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae"}
Jul 15 12:54:57.329185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.329153    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc" event={"ID":"202794ac-3921-4e2d-9cdd-be13b7668444","Type":"ContainerDied","Data":"0b139085a6f4f46df675760678336c98176262501dc680f43fcc9eca82958f99"}
Jul 15 12:54:57.329185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.329175    2576 scope.go:117] "RemoveContainer" containerID="b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae"
Jul 15 12:54:57.338433 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.338419    2576 scope.go:117] "RemoveContainer" containerID="6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"
Jul 15 12:54:57.345878 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.345861    2576 scope.go:117] "RemoveContainer" containerID="b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae"
Jul 15 12:54:57.346115 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:54:57.346094    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae\": container with ID starting with b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae not found: ID does not exist" containerID="b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae"
Jul 15 12:54:57.346171 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.346123    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae"} err="failed to get container status \"b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae\": rpc error: code = NotFound desc = could not find container \"b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae\": container with ID starting with b1848fcf76f93327105ba5b683abc946b4007000316b70fa0924b22b9eef11ae not found: ID does not exist"
Jul 15 12:54:57.346171 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.346146    2576 scope.go:117] "RemoveContainer" containerID="6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"
Jul 15 12:54:57.346421 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:54:57.346405    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0\": container with ID starting with 6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0 not found: ID does not exist" containerID="6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"
Jul 15 12:54:57.346460 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.346427    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0"} err="failed to get container status \"6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0\": rpc error: code = NotFound desc = could not find container \"6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0\": container with ID starting with 6b59064d49cd590a57e5a0aef29ff036230a2dfb4a40abcb948d3e2b693ef9f0 not found: ID does not exist"
Jul 15 12:54:57.360233 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.360210    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"]
Jul 15 12:54:57.365106 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.365085    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1f25dafc-openshift-default-5ddb64995f-kslkc"]
Jul 15 12:54:57.369466 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.369449    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-envoy\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.369521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.369469    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/202794ac-3921-4e2d-9cdd-be13b7668444-istio-podinfo\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.369521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.369477    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-istio-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.369521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.369486    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-workload-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.369521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.369493    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/202794ac-3921-4e2d-9cdd-be13b7668444-istio-data\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.369521 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.369501    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9gmc\" (UniqueName: \"kubernetes.io/projected/202794ac-3921-4e2d-9cdd-be13b7668444-kube-api-access-l9gmc\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:54:57.572058 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:54:57.571979    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" path="/var/lib/kubelet/pods/202794ac-3921-4e2d-9cdd-be13b7668444/volumes"
Jul 15 12:55:11.622474 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:11.622439    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:55:11.625577 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:11.625559    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 12:55:52.018239 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018208    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"]
Jul 15 12:55:52.018726 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018625    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy"
Jul 15 12:55:52.018726 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018637    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy"
Jul 15 12:55:52.018726 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018671    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="722e869e-f1d2-4cfe-9439-327c11873dbb" containerName="maas-api"
Jul 15 12:55:52.018726 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018678    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="722e869e-f1d2-4cfe-9439-327c11873dbb" containerName="maas-api"
Jul 15 12:55:52.018862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018739    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy"
Jul 15 12:55:52.018862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018749    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy"
Jul 15 12:55:52.018862 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.018756    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="722e869e-f1d2-4cfe-9439-327c11873dbb" containerName="maas-api"
Jul 15 12:55:52.021906 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.021886    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.024975 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.024950    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4060a9-serving-cert\""
Jul 15 12:55:52.031636 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.031614    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"]
Jul 15 12:55:52.085359 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.085324    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhnlq\" (UniqueName: \"kubernetes.io/projected/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-kube-api-access-hhnlq\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.085552 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.085399    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.085552 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.085451    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.187025 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.186999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.187217 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.187164    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhnlq\" (UniqueName: \"kubernetes.io/projected/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-kube-api-access-hhnlq\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.187280 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.187223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.187366 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:55:52.187353    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-serving-cert: secret "maas-api-e2e-inf-a-4060a9-serving-cert" not found
Jul 15 12:55:52.187422 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:55:52.187409    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls podName:4bc8b2ac-8563-4eb9-a305-aa9b6a82b919 nodeName:}" failed. No retries permitted until 2026-07-15 12:55:52.687390865 +0000 UTC m=+2141.564113853 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls") pod "maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" (UID: "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919") : secret "maas-api-e2e-inf-a-4060a9-serving-cert" not found
Jul 15 12:55:52.187739 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.187721    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.202617 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.202590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhnlq\" (UniqueName: \"kubernetes.io/projected/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-kube-api-access-hhnlq\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.692449 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.692409    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.694755 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.694727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls\") pod \"maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:52.932248 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:52.932210    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:53.301013 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:53.300987    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"]
Jul 15 12:55:53.303216 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:55:53.303181    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc8b2ac_8563_4eb9_a305_aa9b6a82b919.slice/crio-3fe69e64177d0363619be195a908014d539db068a75ef410e280cb267d3d228d WatchSource:0}: Error finding container 3fe69e64177d0363619be195a908014d539db068a75ef410e280cb267d3d228d: Status 404 returned error can't find the container with id 3fe69e64177d0363619be195a908014d539db068a75ef410e280cb267d3d228d
Jul 15 12:55:53.584124 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:53.584031    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" event={"ID":"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919","Type":"ContainerStarted","Data":"3fe69e64177d0363619be195a908014d539db068a75ef410e280cb267d3d228d"}
Jul 15 12:55:54.591120 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:54.591084    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" event={"ID":"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919","Type":"ContainerStarted","Data":"058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd"}
Jul 15 12:55:54.591491 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:54.591206    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:55:54.626002 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:54.625955    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" podStartSLOduration=3.126592923 podStartE2EDuration="3.625942279s" totalImagesPullingTime="499.349356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:55:54.62436575 +0000 UTC m=+2143.501088755" watchObservedRunningTime="2026-07-15 12:55:54.625942279 +0000 UTC m=+2143.502665336"
Jul 15 12:55:57.149655 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.149625    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"]
Jul 15 12:55:57.150245 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.150224    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy"
Jul 15 12:55:57.150245 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.150246    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="202794ac-3921-4e2d-9cdd-be13b7668444" containerName="istio-proxy"
Jul 15 12:55:57.153584 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.153563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.158807 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.158783    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ee3fe5-openshift-default-dockercfg-9bvn5\""
Jul 15 12:55:57.184127 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.184104    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"]
Jul 15 12:55:57.228077 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228045    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-socket\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228077 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228077    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-credential-socket\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228232 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-data\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228232 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228171    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-podinfo\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228232 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228191    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istiod-ca-cert\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228336 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228252    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7w8q\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-kube-api-access-q7w8q\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228336 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-certs\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228410 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228341    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-token\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.228410 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.228371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-envoy\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329179 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-data\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329179 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-podinfo\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istiod-ca-cert\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329224    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7w8q\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-kube-api-access-q7w8q\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-certs\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329289    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-token\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329319    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-envoy\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329378    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-socket\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329427 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329403    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-credential-socket\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.329985 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329955    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-data\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.330168 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.330131    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-socket\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.330168 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.329969    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istiod-ca-cert\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.330352 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.330021    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-certs\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.330352 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.330184    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-credential-socket\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.331810 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.331791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-envoy\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.332061 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.332043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-podinfo\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.339918 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.339882    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7w8q\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-kube-api-access-q7w8q\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.341504 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.341486    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-token\") pod \"e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") " pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.463680 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.463608    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:57.616396 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.616369    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"]
Jul 15 12:55:57.616841 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.616812    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:55:57.617027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.616880    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:55:57.617027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:57.616920    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:55:58.612338 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:58.612304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" event={"ID":"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6","Type":"ContainerStarted","Data":"6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"}
Jul 15 12:55:58.612732 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:58.612354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" event={"ID":"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6","Type":"ContainerStarted","Data":"0352cdbeaa49116411a1b641169284105a01dc9b251fdaf7019a5602ec9b2f67"}
Jul 15 12:55:58.641263 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:58.641210    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podStartSLOduration=1.641194592 podStartE2EDuration="1.641194592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:55:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:55:58.639195865 +0000 UTC m=+2147.515918872" watchObservedRunningTime="2026-07-15 12:55:58.641194592 +0000 UTC m=+2147.517917597"
Jul 15 12:55:59.464384 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:59.464350    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:59.469053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:59.469028    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:59.616765 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:59.616730    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:55:59.617452 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:55:59.617433    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:56:00.600722 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:00.600690    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:56:00.911793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:00.911767    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"]
Jul 15 12:56:00.915559 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:00.915541    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:00.919363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:00.919340    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ee3fe5-serving-cert\""
Jul 15 12:56:00.944157 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:00.944130    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"]
Jul 15 12:56:01.062292 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.062261    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/923db481-5606-427a-b1e2-f5bd2268d8cc-maas-api-tls\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.062292 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.062297    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx4x2\" (UniqueName: \"kubernetes.io/projected/923db481-5606-427a-b1e2-f5bd2268d8cc-kube-api-access-nx4x2\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.062490 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.062417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/923db481-5606-427a-b1e2-f5bd2268d8cc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.163152 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.163075    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/923db481-5606-427a-b1e2-f5bd2268d8cc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.163279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.163149    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/923db481-5606-427a-b1e2-f5bd2268d8cc-maas-api-tls\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.163279 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.163177    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx4x2\" (UniqueName: \"kubernetes.io/projected/923db481-5606-427a-b1e2-f5bd2268d8cc-kube-api-access-nx4x2\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.163800 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.163777    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/923db481-5606-427a-b1e2-f5bd2268d8cc-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.165436 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.165416    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/923db481-5606-427a-b1e2-f5bd2268d8cc-maas-api-tls\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.179707 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.179686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx4x2\" (UniqueName: \"kubernetes.io/projected/923db481-5606-427a-b1e2-f5bd2268d8cc-kube-api-access-nx4x2\") pod \"maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.227510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.227486    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:01.370051 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.370016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"]
Jul 15 12:56:01.371916 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:56:01.371892    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod923db481_5606_427a_b1e2_f5bd2268d8cc.slice/crio-4864eb3eab9bff49a94dda0013cdb138f2a4c13e45b878ad9c77df810886302b WatchSource:0}: Error finding container 4864eb3eab9bff49a94dda0013cdb138f2a4c13e45b878ad9c77df810886302b: Status 404 returned error can't find the container with id 4864eb3eab9bff49a94dda0013cdb138f2a4c13e45b878ad9c77df810886302b
Jul 15 12:56:01.627818 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:01.627736    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t" event={"ID":"923db481-5606-427a-b1e2-f5bd2268d8cc","Type":"ContainerStarted","Data":"4864eb3eab9bff49a94dda0013cdb138f2a4c13e45b878ad9c77df810886302b"}
Jul 15 12:56:02.633666 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:02.633627    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t" event={"ID":"923db481-5606-427a-b1e2-f5bd2268d8cc","Type":"ContainerStarted","Data":"c7431beb885b90872f3a890be739eb017051043b58720195aa6d0b2d3a65c11b"}
Jul 15 12:56:02.634043 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:02.633783    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:02.661933 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:02.661894    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t" podStartSLOduration=2.276785803 podStartE2EDuration="2.661880261s" totalImagesPullingTime="385.094458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:56:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:56:02.660763362 +0000 UTC m=+2151.537486379" watchObservedRunningTime="2026-07-15 12:56:02.661880261 +0000 UTC m=+2151.538603267"
Jul 15 12:56:08.644430 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:08.644352    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:56:15.459354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.458260    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:39400->10.132.0.56:15021: read: connection reset by peer" start-of-body=
Jul 15 12:56:15.459354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.458322    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:39400->10.132.0.56:15021: read: connection reset by peer"
Jul 15 12:56:15.699227 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.699195    2576 generic.go:356] "Generic (PLEG): container finished" podID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerID="6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579" exitCode=0
Jul 15 12:56:15.699378 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.699261    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" event={"ID":"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6","Type":"ContainerDied","Data":"6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"}
Jul 15 12:56:15.699633 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.699620    2576 scope.go:117] "RemoveContainer" containerID="6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"
Jul 15 12:56:15.700674 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.700647    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:56:15.700786 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.700727    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:56:15.700786 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:15.700771    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:56:16.705653 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:16.705621    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" event={"ID":"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6","Type":"ContainerStarted","Data":"58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e"}
Jul 15 12:56:17.464100 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:17.464071    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:56:17.464298 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:17.464279    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 15 12:56:17.464344 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:17.464323    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 15 12:56:18.464706 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:18.464670    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 15 12:56:18.465072 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:18.464724    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 15 12:56:19.464383 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:19.464347    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 15 12:56:19.464578 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:19.464401    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 15 12:56:20.464262 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:20.464226    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 15 12:56:20.464663 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:20.464282    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 15 12:56:21.464845 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:21.464811    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 15 12:56:21.465213 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:21.464868    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 15 12:56:22.468056 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:22.468027    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:56:22.468471 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:22.468296    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:56:22.468998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:56:22.468979    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:57:25.941708 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:25.941680    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"]
Jul 15 12:57:25.942242 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:25.941878    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" podUID="4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" containerName="maas-api" containerID="cri-o://058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd" gracePeriod=30
Jul 15 12:57:26.188314 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.188287    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:57:26.363941 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.363872    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhnlq\" (UniqueName: \"kubernetes.io/projected/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-kube-api-access-hhnlq\") pod \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") "
Jul 15 12:57:26.363941 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.363908    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls\") pod \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") "
Jul 15 12:57:26.364103 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.364003    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-openshift-service-ca\") pod \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\" (UID: \"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919\") "
Jul 15 12:57:26.364386 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.364365    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" (UID: "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:57:26.365945 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.365924    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" (UID: "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:57:26.366027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.365926    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-kube-api-access-hhnlq" (OuterVolumeSpecName: "kube-api-access-hhnlq") pod "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" (UID: "4bc8b2ac-8563-4eb9-a305-aa9b6a82b919"). InnerVolumeSpecName "kube-api-access-hhnlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:57:26.464775 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.464749    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhnlq\" (UniqueName: \"kubernetes.io/projected/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-kube-api-access-hhnlq\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:57:26.464775 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.464772    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:57:26.464920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:26.464781    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:57:27.044551 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.044495    2576 generic.go:356] "Generic (PLEG): container finished" podID="4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" containerID="058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd" exitCode=0
Jul 15 12:57:27.045027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.044561    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"
Jul 15 12:57:27.045027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.044588    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" event={"ID":"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919","Type":"ContainerDied","Data":"058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd"}
Jul 15 12:57:27.045027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.044649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j" event={"ID":"4bc8b2ac-8563-4eb9-a305-aa9b6a82b919","Type":"ContainerDied","Data":"3fe69e64177d0363619be195a908014d539db068a75ef410e280cb267d3d228d"}
Jul 15 12:57:27.045027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.044680    2576 scope.go:117] "RemoveContainer" containerID="058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd"
Jul 15 12:57:27.053913 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.053895    2576 scope.go:117] "RemoveContainer" containerID="058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd"
Jul 15 12:57:27.054154 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:57:27.054135    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd\": container with ID starting with 058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd not found: ID does not exist" containerID="058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd"
Jul 15 12:57:27.054226 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.054161    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd"} err="failed to get container status \"058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd\": rpc error: code = NotFound desc = could not find container \"058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd\": container with ID starting with 058963f6a7065f2489a8ec95782b5053c747fb9da4e2148530de5cb99e0bcccd not found: ID does not exist"
Jul 15 12:57:27.069393 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.069368    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"]
Jul 15 12:57:27.075851 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.075829    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4060a9-7f95f64f98-ks99j"]
Jul 15 12:57:27.572430 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:27.572393    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" path="/var/lib/kubelet/pods/4bc8b2ac-8563-4eb9-a305-aa9b6a82b919/volumes"
Jul 15 12:57:52.044466 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.044388    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"]
Jul 15 12:57:52.044930 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.044608    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t" podUID="923db481-5606-427a-b1e2-f5bd2268d8cc" containerName="maas-api" containerID="cri-o://c7431beb885b90872f3a890be739eb017051043b58720195aa6d0b2d3a65c11b" gracePeriod=30
Jul 15 12:57:52.161424 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.161396    2576 generic.go:356] "Generic (PLEG): container finished" podID="923db481-5606-427a-b1e2-f5bd2268d8cc" containerID="c7431beb885b90872f3a890be739eb017051043b58720195aa6d0b2d3a65c11b" exitCode=0
Jul 15 12:57:52.161584 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.161462    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t" event={"ID":"923db481-5606-427a-b1e2-f5bd2268d8cc","Type":"ContainerDied","Data":"c7431beb885b90872f3a890be739eb017051043b58720195aa6d0b2d3a65c11b"}
Jul 15 12:57:52.292464 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.292442    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:57:52.377185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.377120    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/923db481-5606-427a-b1e2-f5bd2268d8cc-openshift-service-ca\") pod \"923db481-5606-427a-b1e2-f5bd2268d8cc\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") "
Jul 15 12:57:52.377185 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.377164    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx4x2\" (UniqueName: \"kubernetes.io/projected/923db481-5606-427a-b1e2-f5bd2268d8cc-kube-api-access-nx4x2\") pod \"923db481-5606-427a-b1e2-f5bd2268d8cc\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") "
Jul 15 12:57:52.377351 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.377194    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/923db481-5606-427a-b1e2-f5bd2268d8cc-maas-api-tls\") pod \"923db481-5606-427a-b1e2-f5bd2268d8cc\" (UID: \"923db481-5606-427a-b1e2-f5bd2268d8cc\") "
Jul 15 12:57:52.377510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.377456    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/923db481-5606-427a-b1e2-f5bd2268d8cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "923db481-5606-427a-b1e2-f5bd2268d8cc" (UID: "923db481-5606-427a-b1e2-f5bd2268d8cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:57:52.379184 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.379162    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/923db481-5606-427a-b1e2-f5bd2268d8cc-kube-api-access-nx4x2" (OuterVolumeSpecName: "kube-api-access-nx4x2") pod "923db481-5606-427a-b1e2-f5bd2268d8cc" (UID: "923db481-5606-427a-b1e2-f5bd2268d8cc"). InnerVolumeSpecName "kube-api-access-nx4x2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:57:52.379184 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.379173    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/923db481-5606-427a-b1e2-f5bd2268d8cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "923db481-5606-427a-b1e2-f5bd2268d8cc" (UID: "923db481-5606-427a-b1e2-f5bd2268d8cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:57:52.478131 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.478105    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/923db481-5606-427a-b1e2-f5bd2268d8cc-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:57:52.478131 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.478130    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx4x2\" (UniqueName: \"kubernetes.io/projected/923db481-5606-427a-b1e2-f5bd2268d8cc-kube-api-access-nx4x2\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:57:52.478277 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:52.478142    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/923db481-5606-427a-b1e2-f5bd2268d8cc-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:57:53.168403 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:53.168378    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"
Jul 15 12:57:53.168833 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:53.168423    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t" event={"ID":"923db481-5606-427a-b1e2-f5bd2268d8cc","Type":"ContainerDied","Data":"4864eb3eab9bff49a94dda0013cdb138f2a4c13e45b878ad9c77df810886302b"}
Jul 15 12:57:53.168833 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:53.168488    2576 scope.go:117] "RemoveContainer" containerID="c7431beb885b90872f3a890be739eb017051043b58720195aa6d0b2d3a65c11b"
Jul 15 12:57:53.195682 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:53.195662    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"]
Jul 15 12:57:53.203546 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:53.198728    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ee3fe5-6d65597d64-48p9t"]
Jul 15 12:57:53.572041 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:57:53.571962    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="923db481-5606-427a-b1e2-f5bd2268d8cc" path="/var/lib/kubelet/pods/923db481-5606-427a-b1e2-f5bd2268d8cc/volumes"
Jul 15 12:58:07.269789 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:07.269756    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"]
Jul 15 12:58:07.270307 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:07.269975    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" containerID="cri-o://58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e" gracePeriod=30
Jul 15 12:58:07.470696 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:07.470660    2576 patch_prober.go:28] interesting pod/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 15 12:58:07.470844 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:07.470738    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 15 12:58:12.182630 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.182595    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"]
Jul 15 12:58:12.183063 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.182977    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" containerName="maas-api"
Jul 15 12:58:12.183063 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.182987    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" containerName="maas-api"
Jul 15 12:58:12.183063 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.182996    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="923db481-5606-427a-b1e2-f5bd2268d8cc" containerName="maas-api"
Jul 15 12:58:12.183063 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.183001    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="923db481-5606-427a-b1e2-f5bd2268d8cc" containerName="maas-api"
Jul 15 12:58:12.183063 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.183056    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bc8b2ac-8563-4eb9-a305-aa9b6a82b919" containerName="maas-api"
Jul 15 12:58:12.183063 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.183064    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="923db481-5606-427a-b1e2-f5bd2268d8cc" containerName="maas-api"
Jul 15 12:58:12.187786 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.187766    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.191430 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.191411    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-d0ed97-serving-cert\""
Jul 15 12:58:12.216128 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.216105    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"]
Jul 15 12:58:12.240541 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.240509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chbvl\" (UniqueName: \"kubernetes.io/projected/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-kube-api-access-chbvl\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.240638 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.240571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.240638 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.240623    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.341863 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.341835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.341986 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.341960    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chbvl\" (UniqueName: \"kubernetes.io/projected/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-kube-api-access-chbvl\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.342050 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.341996    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.342123 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:58:12.342105    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-serving-cert: secret "maas-api-e2e-api-a-d0ed97-serving-cert" not found
Jul 15 12:58:12.342194 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:58:12.342185    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls podName:2ab6fb5a-40a7-42b4-a12c-612cbedeb64c nodeName:}" failed. No retries permitted until 2026-07-15 12:58:12.842163599 +0000 UTC m=+2281.718886591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls") pod "maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" (UID: "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c") : secret "maas-api-e2e-api-a-d0ed97-serving-cert" not found
Jul 15 12:58:12.342407 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.342386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-openshift-service-ca\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.361341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.361318    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chbvl\" (UniqueName: \"kubernetes.io/projected/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-kube-api-access-chbvl\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.530861 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.530833    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:58:12.645302 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645275    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-certs\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645327    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-token\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645352    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-credential-socket\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645378    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-envoy\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645408    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7w8q\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-kube-api-access-q7w8q\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645433    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-data\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645454    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-podinfo\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645494    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istiod-ca-cert\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.645583 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-socket\") pod \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\" (UID: \"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6\") "
Jul 15 12:58:12.646036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645609    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:58:12.646036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.645867    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:58:12.646036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.646016    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-certs\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.646036 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.646035    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-workload-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.646352 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.646323    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-data" (OuterVolumeSpecName: "istio-data") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:58:12.646469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.646447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:58:12.647014 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.646994    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:58:12.647667 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.647646    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 12:58:12.647798 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.647779    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-kube-api-access-q7w8q" (OuterVolumeSpecName: "kube-api-access-q7w8q") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "kube-api-access-q7w8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:58:12.647932 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.647912    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-token" (OuterVolumeSpecName: "istio-token") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:58:12.647932 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.647916    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" (UID: "747188fc-1d2b-41e7-8dc9-0e3f6343e8b6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:58:12.747053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.746997    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7w8q\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-kube-api-access-q7w8q\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.747053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.747019    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-data\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.747053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.747029    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-podinfo\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.747053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.747036    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istiod-ca-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.747053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.747044    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.747053 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.747052    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-credential-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.747306 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.747060    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6-istio-envoy\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:12.847562 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.847518    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:12.849696 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:12.849681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls\") pod \"maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:13.097591 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.097497    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:13.268716 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.268670    2576 generic.go:356] "Generic (PLEG): container finished" podID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerID="58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e" exitCode=0
Jul 15 12:58:13.269087 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.268731    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"
Jul 15 12:58:13.269087 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.268765    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" event={"ID":"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6","Type":"ContainerDied","Data":"58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e"}
Jul 15 12:58:13.269087 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.268825    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh" event={"ID":"747188fc-1d2b-41e7-8dc9-0e3f6343e8b6","Type":"ContainerDied","Data":"0352cdbeaa49116411a1b641169284105a01dc9b251fdaf7019a5602ec9b2f67"}
Jul 15 12:58:13.269087 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.268853    2576 scope.go:117] "RemoveContainer" containerID="58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e"
Jul 15 12:58:13.277331 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.277316    2576 scope.go:117] "RemoveContainer" containerID="6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"
Jul 15 12:58:13.284838 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.284821    2576 scope.go:117] "RemoveContainer" containerID="58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e"
Jul 15 12:58:13.285094 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:58:13.285078    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e\": container with ID starting with 58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e not found: ID does not exist" containerID="58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e"
Jul 15 12:58:13.285147 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.285101    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e"} err="failed to get container status \"58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e\": rpc error: code = NotFound desc = could not find container \"58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e\": container with ID starting with 58d69b5393df005e9a9bd86bc37feb8915d7563574e2581e5645b1ec0e00d52e not found: ID does not exist"
Jul 15 12:58:13.285147 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.285118    2576 scope.go:117] "RemoveContainer" containerID="6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"
Jul 15 12:58:13.285354 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:58:13.285339    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579\": container with ID starting with 6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579 not found: ID does not exist" containerID="6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"
Jul 15 12:58:13.285394 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.285369    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579"} err="failed to get container status \"6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579\": rpc error: code = NotFound desc = could not find container \"6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579\": container with ID starting with 6c541a60cd49c8eaf5eb5dfc3cdcd79167c18b7be435b6df2bc6f3b62e652579 not found: ID does not exist"
Jul 15 12:58:13.355303 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.355239    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"]
Jul 15 12:58:13.360928 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.360906    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ee3fe5-openshift-default-56cdfd5d6f-87sjh"]
Jul 15 12:58:13.372354 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.372314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"]
Jul 15 12:58:13.374462 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:58:13.374438    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ab6fb5a_40a7_42b4_a12c_612cbedeb64c.slice/crio-3b8b342d9841afa16ed9076ceb9f333142db6ea5c721efdbeafc931a09fb43bd WatchSource:0}: Error finding container 3b8b342d9841afa16ed9076ceb9f333142db6ea5c721efdbeafc931a09fb43bd: Status 404 returned error can't find the container with id 3b8b342d9841afa16ed9076ceb9f333142db6ea5c721efdbeafc931a09fb43bd
Jul 15 12:58:13.375858 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.375840    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 15 12:58:13.571607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:13.571578    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" path="/var/lib/kubelet/pods/747188fc-1d2b-41e7-8dc9-0e3f6343e8b6/volumes"
Jul 15 12:58:14.275610 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:14.275572    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" event={"ID":"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c","Type":"ContainerStarted","Data":"b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23"}
Jul 15 12:58:14.275987 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:14.275656    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" event={"ID":"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c","Type":"ContainerStarted","Data":"3b8b342d9841afa16ed9076ceb9f333142db6ea5c721efdbeafc931a09fb43bd"}
Jul 15 12:58:14.275987 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:14.275718    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:14.314146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:14.314105    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" podStartSLOduration=1.8263262679999999 podStartE2EDuration="2.314092462s" totalImagesPullingTime="487.766194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:58:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:58:14.313299251 +0000 UTC m=+2283.190022269" watchObservedRunningTime="2026-07-15 12:58:14.314092462 +0000 UTC m=+2283.190815468"
Jul 15 12:58:17.524482 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.524447    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"]
Jul 15 12:58:17.525027 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.525011    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy"
Jul 15 12:58:17.525071 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.525030    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy"
Jul 15 12:58:17.525150 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.525139    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy"
Jul 15 12:58:17.528483 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.528454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.545295 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.545274    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-aed9ee-openshift-default-dockercfg-cxxh8\""
Jul 15 12:58:17.590203 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590175    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-data\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590311 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590230    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-credential-socket\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590311 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590260    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-envoy\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590311 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590286    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-podinfo\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590458 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590352    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istiod-ca-cert\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590458 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590391    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-token\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590458 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590417    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-socket\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590458 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590432    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-certs\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.590670 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.590595    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cw4t\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-kube-api-access-9cw4t\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.669590 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.669561    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"]
Jul 15 12:58:17.691385 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691355    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-credential-socket\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691505 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691399    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-envoy\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691505 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-podinfo\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691505 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istiod-ca-cert\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691563    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-token\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-socket\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691661 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-certs\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691809 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691729    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cw4t\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-kube-api-access-9cw4t\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691809 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-credential-socket\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.691809 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.691780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-data\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.692037 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.692018    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-data\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.692189 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.692144    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-socket\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.692189 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.692163    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istiod-ca-cert\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.692295 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.692194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-certs\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.693763 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.693743    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-envoy\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.693888 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.693872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-podinfo\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.710835 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.710810    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cw4t\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-kube-api-access-9cw4t\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.711147 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.711126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-token\") pod \"e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") " pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.839133 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.839062    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:17.984010 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.983981    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"]
Jul 15 12:58:17.986397 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:58:17.986373    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ef571a3_e821_4ac7_89c3_af53c5019fcd.slice/crio-ab791f91e584f61a1ffa2d26ae63635fbc0b04fc46181e00de3b22e1a3514304 WatchSource:0}: Error finding container ab791f91e584f61a1ffa2d26ae63635fbc0b04fc46181e00de3b22e1a3514304: Status 404 returned error can't find the container with id ab791f91e584f61a1ffa2d26ae63635fbc0b04fc46181e00de3b22e1a3514304
Jul 15 12:58:17.990196 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.989379    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:58:17.990196 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.989612    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:58:17.990196 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:17.989705    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:58:18.296920 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:18.296885    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" event={"ID":"1ef571a3-e821-4ac7-89c3-af53c5019fcd","Type":"ContainerStarted","Data":"99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"}
Jul 15 12:58:18.297108 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:18.296965    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" event={"ID":"1ef571a3-e821-4ac7-89c3-af53c5019fcd","Type":"ContainerStarted","Data":"ab791f91e584f61a1ffa2d26ae63635fbc0b04fc46181e00de3b22e1a3514304"}
Jul 15 12:58:18.331369 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:18.331318    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" podStartSLOduration=1.33130398 podStartE2EDuration="1.33130398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:58:18.327611193 +0000 UTC m=+2287.204334201" watchObservedRunningTime="2026-07-15 12:58:18.33130398 +0000 UTC m=+2287.208026986"
Jul 15 12:58:18.839155 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:18.839120    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:19.844496 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:19.844466    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:20.285716 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.285686    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:20.306594 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.306567    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:20.307433 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.307416    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:20.499396 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.499352    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"]
Jul 15 12:58:20.499794 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.499781    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy"
Jul 15 12:58:20.499838 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.499796    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy"
Jul 15 12:58:20.499887 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.499878    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="747188fc-1d2b-41e7-8dc9-0e3f6343e8b6" containerName="istio-proxy"
Jul 15 12:58:20.503177 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.503152    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.506125 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.506102    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-aed9ee-serving-cert\""
Jul 15 12:58:20.519953 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.519931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"]
Jul 15 12:58:20.619387 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.619303    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c1872cf-3841-4c62-b70c-1309f451586d-maas-api-tls\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.619544 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.619428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c1872cf-3841-4c62-b70c-1309f451586d-openshift-service-ca\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.619544 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.619454    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6hx5\" (UniqueName: \"kubernetes.io/projected/0c1872cf-3841-4c62-b70c-1309f451586d-kube-api-access-v6hx5\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.720420 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.720384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c1872cf-3841-4c62-b70c-1309f451586d-maas-api-tls\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.720618 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.720505    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c1872cf-3841-4c62-b70c-1309f451586d-openshift-service-ca\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.720618 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.720560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6hx5\" (UniqueName: \"kubernetes.io/projected/0c1872cf-3841-4c62-b70c-1309f451586d-kube-api-access-v6hx5\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.721127 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.721105    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c1872cf-3841-4c62-b70c-1309f451586d-openshift-service-ca\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.722745 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.722728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c1872cf-3841-4c62-b70c-1309f451586d-maas-api-tls\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.732662 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.732638    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6hx5\" (UniqueName: \"kubernetes.io/projected/0c1872cf-3841-4c62-b70c-1309f451586d-kube-api-access-v6hx5\") pod \"maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.814640 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.814609    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:20.953409 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:20.953381    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"]
Jul 15 12:58:20.954572 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:58:20.954525    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c1872cf_3841_4c62_b70c_1309f451586d.slice/crio-b4dd2d90e45a7198f1f40452cff21a8845330896fa3941f8da84680564a8a51b WatchSource:0}: Error finding container b4dd2d90e45a7198f1f40452cff21a8845330896fa3941f8da84680564a8a51b: Status 404 returned error can't find the container with id b4dd2d90e45a7198f1f40452cff21a8845330896fa3941f8da84680564a8a51b
Jul 15 12:58:21.311923 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:21.311887    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6" event={"ID":"0c1872cf-3841-4c62-b70c-1309f451586d","Type":"ContainerStarted","Data":"b4dd2d90e45a7198f1f40452cff21a8845330896fa3941f8da84680564a8a51b"}
Jul 15 12:58:22.318309 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:22.318259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6" event={"ID":"0c1872cf-3841-4c62-b70c-1309f451586d","Type":"ContainerStarted","Data":"d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f"}
Jul 15 12:58:22.318684 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:22.318525    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:22.338699 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:22.338650    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6" podStartSLOduration=1.926475993 podStartE2EDuration="2.338633684s" totalImagesPullingTime="412.157691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:58:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:58:22.338088389 +0000 UTC m=+2291.214811413" watchObservedRunningTime="2026-07-15 12:58:22.338633684 +0000 UTC m=+2291.215356692"
Jul 15 12:58:28.330976 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:28.330936    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:58:33.374684 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:33.374606    2576 generic.go:356] "Generic (PLEG): container finished" podID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerID="99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1" exitCode=0
Jul 15 12:58:33.374684 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:33.374648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" event={"ID":"1ef571a3-e821-4ac7-89c3-af53c5019fcd","Type":"ContainerDied","Data":"99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"}
Jul 15 12:58:33.375085 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:33.374993    2576 scope.go:117] "RemoveContainer" containerID="99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"
Jul 15 12:58:33.375976 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:33.375946    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:58:33.376102 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:33.376027    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:58:33.376102 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:33.376075    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:58:34.387604 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:34.387570    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" event={"ID":"1ef571a3-e821-4ac7-89c3-af53c5019fcd","Type":"ContainerStarted","Data":"0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3"}
Jul 15 12:58:34.840371 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:34.840330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:34.841628 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:34.841594    2576 patch_prober.go:28] interesting pod/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 15 12:58:34.841757 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:34.841654    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 15 12:58:35.307564 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:35.307517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:35.840153 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:35.840117    2576 patch_prober.go:28] interesting pod/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 15 12:58:35.840748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:35.840711    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 15 12:58:36.840194 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:36.840154    2576 patch_prober.go:28] interesting pod/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 15 12:58:36.840793 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:36.840761    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 15 12:58:37.839760 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:37.839724    2576 patch_prober.go:28] interesting pod/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 15 12:58:37.840126 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:37.840096    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 15 12:58:38.843941 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:38.843915    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:38.844981 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:38.844964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:58:43.713552 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:43.713500    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"]
Jul 15 12:58:43.714043 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:43.713723    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" podUID="2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" containerName="maas-api" containerID="cri-o://b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23" gracePeriod=30
Jul 15 12:58:43.980178 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:43.980157    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:44.022713 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.022684    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls\") pod \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") "
Jul 15 12:58:44.022857 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.022747    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-openshift-service-ca\") pod \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") "
Jul 15 12:58:44.022857 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.022834    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chbvl\" (UniqueName: \"kubernetes.io/projected/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-kube-api-access-chbvl\") pod \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\" (UID: \"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c\") "
Jul 15 12:58:44.023106 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.023069    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" (UID: "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:58:44.024849 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.024822    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" (UID: "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:58:44.025026 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.025006    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-kube-api-access-chbvl" (OuterVolumeSpecName: "kube-api-access-chbvl") pod "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" (UID: "2ab6fb5a-40a7-42b4-a12c-612cbedeb64c"). InnerVolumeSpecName "kube-api-access-chbvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:58:44.124239 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.124217    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:44.124341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.124238    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chbvl\" (UniqueName: \"kubernetes.io/projected/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-kube-api-access-chbvl\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:44.124341 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.124258    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:58:44.435227 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.435191    2576 generic.go:356] "Generic (PLEG): container finished" podID="2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" containerID="b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23" exitCode=0
Jul 15 12:58:44.435397 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.435241    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"
Jul 15 12:58:44.435397 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.435278    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" event={"ID":"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c","Type":"ContainerDied","Data":"b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23"}
Jul 15 12:58:44.435397 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.435341    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw" event={"ID":"2ab6fb5a-40a7-42b4-a12c-612cbedeb64c","Type":"ContainerDied","Data":"3b8b342d9841afa16ed9076ceb9f333142db6ea5c721efdbeafc931a09fb43bd"}
Jul 15 12:58:44.435397 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.435370    2576 scope.go:117] "RemoveContainer" containerID="b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23"
Jul 15 12:58:44.444912 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.444891    2576 scope.go:117] "RemoveContainer" containerID="b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23"
Jul 15 12:58:44.445153 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:58:44.445134    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23\": container with ID starting with b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23 not found: ID does not exist" containerID="b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23"
Jul 15 12:58:44.445199 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.445161    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23"} err="failed to get container status \"b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23\": rpc error: code = NotFound desc = could not find container \"b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23\": container with ID starting with b78b88b3f391ae9acd89537954506001dfc47ffd4a4dcf1609d6a22ebbd27d23 not found: ID does not exist"
Jul 15 12:58:44.464675 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.464655    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"]
Jul 15 12:58:44.469697 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:44.469675    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-d0ed97-67796695cd-dwdjw"]
Jul 15 12:58:45.571660 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:58:45.571631    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" path="/var/lib/kubelet/pods/2ab6fb5a-40a7-42b4-a12c-612cbedeb64c/volumes"
Jul 15 12:59:09.938721 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:09.938511    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"]
Jul 15 12:59:09.939159 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:09.938825    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6" podUID="0c1872cf-3841-4c62-b70c-1309f451586d" containerName="maas-api" containerID="cri-o://d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f" gracePeriod=30
Jul 15 12:59:10.193332 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.193273    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:59:10.350614 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.350588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c1872cf-3841-4c62-b70c-1309f451586d-openshift-service-ca\") pod \"0c1872cf-3841-4c62-b70c-1309f451586d\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") "
Jul 15 12:59:10.350768 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.350649    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c1872cf-3841-4c62-b70c-1309f451586d-maas-api-tls\") pod \"0c1872cf-3841-4c62-b70c-1309f451586d\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") "
Jul 15 12:59:10.350829 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.350812    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6hx5\" (UniqueName: \"kubernetes.io/projected/0c1872cf-3841-4c62-b70c-1309f451586d-kube-api-access-v6hx5\") pod \"0c1872cf-3841-4c62-b70c-1309f451586d\" (UID: \"0c1872cf-3841-4c62-b70c-1309f451586d\") "
Jul 15 12:59:10.350971 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.350948    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c1872cf-3841-4c62-b70c-1309f451586d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0c1872cf-3841-4c62-b70c-1309f451586d" (UID: "0c1872cf-3841-4c62-b70c-1309f451586d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:59:10.351095 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.351081    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c1872cf-3841-4c62-b70c-1309f451586d-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:10.352609 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.352588    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c1872cf-3841-4c62-b70c-1309f451586d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0c1872cf-3841-4c62-b70c-1309f451586d" (UID: "0c1872cf-3841-4c62-b70c-1309f451586d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 12:59:10.352703 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.352692    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c1872cf-3841-4c62-b70c-1309f451586d-kube-api-access-v6hx5" (OuterVolumeSpecName: "kube-api-access-v6hx5") pod "0c1872cf-3841-4c62-b70c-1309f451586d" (UID: "0c1872cf-3841-4c62-b70c-1309f451586d"). InnerVolumeSpecName "kube-api-access-v6hx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:59:10.452372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.452304    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6hx5\" (UniqueName: \"kubernetes.io/projected/0c1872cf-3841-4c62-b70c-1309f451586d-kube-api-access-v6hx5\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:10.452372 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.452329    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0c1872cf-3841-4c62-b70c-1309f451586d-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:10.560300 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.560271    2576 generic.go:356] "Generic (PLEG): container finished" podID="0c1872cf-3841-4c62-b70c-1309f451586d" containerID="d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f" exitCode=0
Jul 15 12:59:10.560431 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.560328    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"
Jul 15 12:59:10.560431 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.560354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6" event={"ID":"0c1872cf-3841-4c62-b70c-1309f451586d","Type":"ContainerDied","Data":"d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f"}
Jul 15 12:59:10.560431 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.560424    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6" event={"ID":"0c1872cf-3841-4c62-b70c-1309f451586d","Type":"ContainerDied","Data":"b4dd2d90e45a7198f1f40452cff21a8845330896fa3941f8da84680564a8a51b"}
Jul 15 12:59:10.560545 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.560446    2576 scope.go:117] "RemoveContainer" containerID="d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f"
Jul 15 12:59:10.569041 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.569026    2576 scope.go:117] "RemoveContainer" containerID="d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f"
Jul 15 12:59:10.569291 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:10.569274    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f\": container with ID starting with d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f not found: ID does not exist" containerID="d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f"
Jul 15 12:59:10.569353 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.569299    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f"} err="failed to get container status \"d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f\": rpc error: code = NotFound desc = could not find container \"d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f\": container with ID starting with d8910d09786ec3d742090ef2247453cac06304f899c454e63db95e8180e8d04f not found: ID does not exist"
Jul 15 12:59:10.584498 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.584471    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"]
Jul 15 12:59:10.590435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:10.590415    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-aed9ee-78b5ccf567-49jq6"]
Jul 15 12:59:11.572549 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:11.572510    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c1872cf-3841-4c62-b70c-1309f451586d" path="/var/lib/kubelet/pods/0c1872cf-3841-4c62-b70c-1309f451586d/volumes"
Jul 15 12:59:25.343729 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:25.343696    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"]
Jul 15 12:59:25.344246 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:25.343893    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy" containerID="cri-o://0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3" gracePeriod=30
Jul 15 12:59:29.552032 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.551999    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"]
Jul 15 12:59:29.552461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.552413    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c1872cf-3841-4c62-b70c-1309f451586d" containerName="maas-api"
Jul 15 12:59:29.552461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.552424    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c1872cf-3841-4c62-b70c-1309f451586d" containerName="maas-api"
Jul 15 12:59:29.552461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.552454    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" containerName="maas-api"
Jul 15 12:59:29.552461 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.552459    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" containerName="maas-api"
Jul 15 12:59:29.552682 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.552513    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ab6fb5a-40a7-42b4-a12c-612cbedeb64c" containerName="maas-api"
Jul 15 12:59:29.552682 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.552524    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c1872cf-3841-4c62-b70c-1309f451586d" containerName="maas-api"
Jul 15 12:59:29.555525 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.555509    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.561179 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.561160    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-3af497-serving-cert\""
Jul 15 12:59:29.573323 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.573302    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"]
Jul 15 12:59:29.718547 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.718504    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92469482-9d9f-49f9-9e9c-d783306fb289-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.718715 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.718669    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdrhd\" (UniqueName: \"kubernetes.io/projected/92469482-9d9f-49f9-9e9c-d783306fb289-kube-api-access-wdrhd\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.718987 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.718963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92469482-9d9f-49f9-9e9c-d783306fb289-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.820269 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.820199    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92469482-9d9f-49f9-9e9c-d783306fb289-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.820269 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.820253    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdrhd\" (UniqueName: \"kubernetes.io/projected/92469482-9d9f-49f9-9e9c-d783306fb289-kube-api-access-wdrhd\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.820423 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.820292    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92469482-9d9f-49f9-9e9c-d783306fb289-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.820873 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.820854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92469482-9d9f-49f9-9e9c-d783306fb289-openshift-service-ca\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.822510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.822489    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92469482-9d9f-49f9-9e9c-d783306fb289-maas-api-tls\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.828860 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.828838    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdrhd\" (UniqueName: \"kubernetes.io/projected/92469482-9d9f-49f9-9e9c-d783306fb289-kube-api-access-wdrhd\") pod \"maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.867209 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.867185    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:29.998975 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:29.998949    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"]
Jul 15 12:59:30.000748 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:59:30.000723    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92469482_9d9f_49f9_9e9c_d783306fb289.slice/crio-aa613828a79ca524b01f55be585a5570efa5c1394da96f9b10720bda59f42305 WatchSource:0}: Error finding container aa613828a79ca524b01f55be585a5570efa5c1394da96f9b10720bda59f42305: Status 404 returned error can't find the container with id aa613828a79ca524b01f55be585a5570efa5c1394da96f9b10720bda59f42305
Jul 15 12:59:30.579640 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.579613    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:59:30.654356 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.654324    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t" event={"ID":"92469482-9d9f-49f9-9e9c-d783306fb289","Type":"ContainerStarted","Data":"aa613828a79ca524b01f55be585a5570efa5c1394da96f9b10720bda59f42305"}
Jul 15 12:59:30.655905 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.655869    2576 generic.go:356] "Generic (PLEG): container finished" podID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerID="0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3" exitCode=0
Jul 15 12:59:30.655905 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.655905    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" event={"ID":"1ef571a3-e821-4ac7-89c3-af53c5019fcd","Type":"ContainerDied","Data":"0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3"}
Jul 15 12:59:30.656082 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.655910    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"
Jul 15 12:59:30.656082 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.655952    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2" event={"ID":"1ef571a3-e821-4ac7-89c3-af53c5019fcd","Type":"ContainerDied","Data":"ab791f91e584f61a1ffa2d26ae63635fbc0b04fc46181e00de3b22e1a3514304"}
Jul 15 12:59:30.656082 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.655984    2576 scope.go:117] "RemoveContainer" containerID="0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3"
Jul 15 12:59:30.663979 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.663965    2576 scope.go:117] "RemoveContainer" containerID="99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"
Jul 15 12:59:30.671595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.671577    2576 scope.go:117] "RemoveContainer" containerID="0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3"
Jul 15 12:59:30.671851 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:30.671827    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3\": container with ID starting with 0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3 not found: ID does not exist" containerID="0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3"
Jul 15 12:59:30.671922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.671862    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3"} err="failed to get container status \"0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3\": rpc error: code = NotFound desc = could not find container \"0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3\": container with ID starting with 0133bf0370d35dd4c2e0ea7f19ff921432e4fe44b280299ee9e73ad4d1b88ee3 not found: ID does not exist"
Jul 15 12:59:30.671922 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.671885    2576 scope.go:117] "RemoveContainer" containerID="99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"
Jul 15 12:59:30.672132 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:30.672106    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1\": container with ID starting with 99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1 not found: ID does not exist" containerID="99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"
Jul 15 12:59:30.672173 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.672138    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1"} err="failed to get container status \"99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1\": rpc error: code = NotFound desc = could not find container \"99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1\": container with ID starting with 99345913269fdbd2586e7472380fd976c200614e309ad1f728ec8818a1fc90c1 not found: ID does not exist"
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730614    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-podinfo\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730666    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-token\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-socket\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730739    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-certs\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730776    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istiod-ca-cert\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730946    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-envoy\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.730972    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cw4t\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-kube-api-access-9cw4t\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-credential-socket\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731036    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731068    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-data\") pod \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\" (UID: \"1ef571a3-e821-4ac7-89c3-af53c5019fcd\") "
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731202    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731402    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-certs\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731420    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istiod-ca-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.731486 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731416    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:59:30.732237 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.731571    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:59:30.732369 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.732347    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:59:30.733336 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.733308    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 12:59:30.733985 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.733808    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 12:59:30.733985 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.733968    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-kube-api-access-9cw4t" (OuterVolumeSpecName: "kube-api-access-9cw4t") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "kube-api-access-9cw4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:59:30.734098 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.734048    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ef571a3-e821-4ac7-89c3-af53c5019fcd" (UID: "1ef571a3-e821-4ac7-89c3-af53c5019fcd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 12:59:30.831991 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.831963    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-data\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.831991 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.831994    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-podinfo\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.832138 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.832005    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.832138 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.832019    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-workload-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.832138 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.832027    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-istio-envoy\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.832138 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.832035    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cw4t\" (UniqueName: \"kubernetes.io/projected/1ef571a3-e821-4ac7-89c3-af53c5019fcd-kube-api-access-9cw4t\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.832138 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.832044    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ef571a3-e821-4ac7-89c3-af53c5019fcd-credential-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 12:59:30.979820 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.979785    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"]
Jul 15 12:59:30.981650 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:30.981626    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-aed9ee-openshift-default-9dd5d7575-dnhw2"]
Jul 15 12:59:31.573868 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:31.573831    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" path="/var/lib/kubelet/pods/1ef571a3-e821-4ac7-89c3-af53c5019fcd/volumes"
Jul 15 12:59:31.662414 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:31.662386    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t" event={"ID":"92469482-9d9f-49f9-9e9c-d783306fb289","Type":"ContainerStarted","Data":"6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3"}
Jul 15 12:59:31.662787 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:31.662503    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:31.684541 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:31.684483    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t" podStartSLOduration=1.6061829159999998 podStartE2EDuration="2.684466352s" totalImagesPullingTime="1.078283436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:59:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:59:31.682597859 +0000 UTC m=+2360.559320866" watchObservedRunningTime="2026-07-15 12:59:31.684466352 +0000 UTC m=+2360.561189357"
Jul 15 12:59:34.552146 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.552118    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"]
Jul 15 12:59:34.552607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.552489    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy"
Jul 15 12:59:34.552607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.552501    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy"
Jul 15 12:59:34.552607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.552510    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy"
Jul 15 12:59:34.552607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.552515    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy"
Jul 15 12:59:34.552607 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.552587    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy"
Jul 15 12:59:34.555748 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.555730    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556362 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556343    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-envoy\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556400    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-credential-socket\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556501 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-socket\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556501 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556455    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-token\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556501 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wwcw\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-kube-api-access-7wwcw\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556681 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556556    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-certs\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556681 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556582    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-data\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556681 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556602    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/952fb2d7-1b83-461a-93d3-a414d6085c8e-istiod-ca-cert\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.556837 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.556673    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-podinfo\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.559268 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.559242    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-70ed7b-openshift-default-dockercfg-68qwm\""
Jul 15 12:59:34.581168 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.581144    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"]
Jul 15 12:59:34.657902 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.657868    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/952fb2d7-1b83-461a-93d3-a414d6085c8e-istiod-ca-cert\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658123 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.657918    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-podinfo\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658123 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.657946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-envoy\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658123 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-credential-socket\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658123 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658060    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-socket\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658123 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658083    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-token\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658123 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwcw\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-kube-api-access-7wwcw\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658481 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658185    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-certs\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.658481 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658226    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-data\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.659150 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658668    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-data\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.659150 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.658885    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-socket\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.659150 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.659040    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/952fb2d7-1b83-461a-93d3-a414d6085c8e-istiod-ca-cert\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.659358 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.659316    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-credential-socket\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.659447 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.659429    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-certs\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.660998 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.660974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-podinfo\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.661200 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.661180    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-envoy\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.686212 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.686194    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-token\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.686618 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.686601    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wwcw\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-kube-api-access-7wwcw\") pod \"e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") " pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:34.866162 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:34.866088    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:35.017497 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.017467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"]
Jul 15 12:59:35.018300 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:59:35.018271    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod952fb2d7_1b83_461a_93d3_a414d6085c8e.slice/crio-1a756d3a2278309d81109453e4bef0c0327304a7881a6ef147246e2ebc11a5bb WatchSource:0}: Error finding container 1a756d3a2278309d81109453e4bef0c0327304a7881a6ef147246e2ebc11a5bb: Status 404 returned error can't find the container with id 1a756d3a2278309d81109453e4bef0c0327304a7881a6ef147246e2ebc11a5bb
Jul 15 12:59:35.020469 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.020439    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:59:35.020548 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.020510    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:59:35.020592 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.020565    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:59:35.682137 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.682104    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" event={"ID":"952fb2d7-1b83-461a-93d3-a414d6085c8e","Type":"ContainerStarted","Data":"22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"}
Jul 15 12:59:35.682510 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.682157    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" event={"ID":"952fb2d7-1b83-461a-93d3-a414d6085c8e","Type":"ContainerStarted","Data":"1a756d3a2278309d81109453e4bef0c0327304a7881a6ef147246e2ebc11a5bb"}
Jul 15 12:59:35.713859 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.713816    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podStartSLOduration=1.7138030199999998 podStartE2EDuration="1.71380302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:59:35.712619416 +0000 UTC m=+2364.589342421" watchObservedRunningTime="2026-07-15 12:59:35.71380302 +0000 UTC m=+2364.590526025"
Jul 15 12:59:35.866374 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:35.866340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:36.871556 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:36.871517    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:37.672005 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:37.671977    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 12:59:37.692785 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:37.692760    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:37.693767 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:37.693743    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:38.064669 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.064593    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"]
Jul 15 12:59:38.065054 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.065040    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ef571a3-e821-4ac7-89c3-af53c5019fcd" containerName="istio-proxy"
Jul 15 12:59:38.068308 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.068291    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.073696 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:38.073669    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-shared-b-70ed7b-serving-cert\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-70ed7b-serving-cert\"" type="*v1.Secret"
Jul 15 12:59:38.076055 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:38.076034    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" is forbidden: User \"system:node:ip-10-0-135-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-135-6.ec2.internal' and this object" podUID="7efb5521-d788-4c31-a53a-8b51c96e1995" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.082321 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.082298    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.082426 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.082344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26jnm\" (UniqueName: \"kubernetes.io/projected/7efb5521-d788-4c31-a53a-8b51c96e1995-kube-api-access-26jnm\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.082492 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.082431    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7efb5521-d788-4c31-a53a-8b51c96e1995-openshift-service-ca\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.100495 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.098573    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"]
Jul 15 12:59:38.183087 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.183055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.183193 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.183111    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26jnm\" (UniqueName: \"kubernetes.io/projected/7efb5521-d788-4c31-a53a-8b51c96e1995-kube-api-access-26jnm\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.183193 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.183170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7efb5521-d788-4c31-a53a-8b51c96e1995-openshift-service-ca\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.183810 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.183789    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7efb5521-d788-4c31-a53a-8b51c96e1995-openshift-service-ca\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:38.205569 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:38.205141    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26jnm\" (UniqueName: \"kubernetes.io/projected/7efb5521-d788-4c31-a53a-8b51c96e1995-kube-api-access-26jnm\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:39.183881 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:39.183849    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-serving-cert: failed to sync secret cache: timed out waiting for the condition
Jul 15 12:59:39.184245 ip-10-0-135-6 kubenswrapper[2576]: E0715 12:59:39.183945    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls podName:7efb5521-d788-4c31-a53a-8b51c96e1995 nodeName:}" failed. No retries permitted until 2026-07-15 12:59:39.683926311 +0000 UTC m=+2368.560649295 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls") pod "maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" (UID: "7efb5521-d788-4c31-a53a-8b51c96e1995") : failed to sync secret cache: timed out waiting for the condition
Jul 15 12:59:39.469102 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:39.469025    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-70ed7b-serving-cert\""
Jul 15 12:59:39.695657 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:39.695626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:39.698057 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:39.698034    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls\") pod \"maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:39.878218 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:39.878141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:40.011364 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:40.011339    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"]
Jul 15 12:59:40.013407 ip-10-0-135-6 kubenswrapper[2576]: W0715 12:59:40.013383    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7efb5521_d788_4c31_a53a_8b51c96e1995.slice/crio-c9f8376ec0dec1994780c773b1c573215f37e853009a2b6dc362f0e3d3917aec WatchSource:0}: Error finding container c9f8376ec0dec1994780c773b1c573215f37e853009a2b6dc362f0e3d3917aec: Status 404 returned error can't find the container with id c9f8376ec0dec1994780c773b1c573215f37e853009a2b6dc362f0e3d3917aec
Jul 15 12:59:40.708363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:40.708271    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" event={"ID":"7efb5521-d788-4c31-a53a-8b51c96e1995","Type":"ContainerStarted","Data":"a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0"}
Jul 15 12:59:40.708363 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:40.708337    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:40.708778 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:40.708364    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" event={"ID":"7efb5521-d788-4c31-a53a-8b51c96e1995","Type":"ContainerStarted","Data":"c9f8376ec0dec1994780c773b1c573215f37e853009a2b6dc362f0e3d3917aec"}
Jul 15 12:59:40.728853 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:40.728801    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" podStartSLOduration=2.3119332249999998 podStartE2EDuration="2.728786077s" totalImagesPullingTime="416.852852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 12:59:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 12:59:40.726678893 +0000 UTC m=+2369.603401904" watchObservedRunningTime="2026-07-15 12:59:40.728786077 +0000 UTC m=+2369.605509083"
Jul 15 12:59:46.719670 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:46.719642    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 12:59:53.734703 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:53.734657    2576 patch_prober.go:28] interesting pod/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 15 12:59:53.735116 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:53.734720    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 15 12:59:55.783026 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:55.782995    2576 generic.go:356] "Generic (PLEG): container finished" podID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerID="22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726" exitCode=0
Jul 15 12:59:55.783557 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:55.783067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" event={"ID":"952fb2d7-1b83-461a-93d3-a414d6085c8e","Type":"ContainerDied","Data":"22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"}
Jul 15 12:59:55.783639 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:55.783574    2576 scope.go:117] "RemoveContainer" containerID="22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"
Jul 15 12:59:55.784506 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:55.784482    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:59:55.784595 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:55.784572    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:59:55.784633 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:55.784618    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 15 12:59:56.789493 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:56.789460    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" event={"ID":"952fb2d7-1b83-461a-93d3-a414d6085c8e","Type":"ContainerStarted","Data":"f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d"}
Jul 15 12:59:56.867142 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:56.867109    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 12:59:56.867377 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:56.867351    2576 patch_prober.go:28] interesting pod/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 15 12:59:56.867435 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:56.867412    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 15 12:59:57.867265 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:57.867230    2576 patch_prober.go:28] interesting pod/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 15 12:59:57.867653 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:57.867281    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 15 12:59:58.866587 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:58.866552    2576 patch_prober.go:28] interesting pod/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 15 12:59:58.866767 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:58.866617    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 15 12:59:59.867282 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:59.867246    2576 patch_prober.go:28] interesting pod/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 15 12:59:59.867709 ip-10-0-135-6 kubenswrapper[2576]: I0715 12:59:59.867304    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 15 13:00:00.870176 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:00:00.870148    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 13:00:00.870647 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:00:00.870411    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 13:00:00.871232 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:00:00.871217    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 13:00:11.650828 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:00:11.650801    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 13:00:11.655626 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:00:11.655609    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 13:05:11.682444 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:05:11.682418    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 13:05:11.687359 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:05:11.687341    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-8fhzw_c87a0dff-b6af-428d-b7fc-0091fa66d911/ovn-acl-logging/0.log"
Jul 15 13:07:08.148383 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.148308    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"]
Jul 15 13:07:08.148852 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.148505    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t" podUID="92469482-9d9f-49f9-9e9c-d783306fb289" containerName="maas-api" containerID="cri-o://6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3" gracePeriod=30
Jul 15 13:07:08.400337 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.400285    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 13:07:08.429826 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.429802    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdrhd\" (UniqueName: \"kubernetes.io/projected/92469482-9d9f-49f9-9e9c-d783306fb289-kube-api-access-wdrhd\") pod \"92469482-9d9f-49f9-9e9c-d783306fb289\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") "
Jul 15 13:07:08.429971 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.429842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92469482-9d9f-49f9-9e9c-d783306fb289-openshift-service-ca\") pod \"92469482-9d9f-49f9-9e9c-d783306fb289\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") "
Jul 15 13:07:08.430059 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.430035    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92469482-9d9f-49f9-9e9c-d783306fb289-maas-api-tls\") pod \"92469482-9d9f-49f9-9e9c-d783306fb289\" (UID: \"92469482-9d9f-49f9-9e9c-d783306fb289\") "
Jul 15 13:07:08.430148 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.430131    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92469482-9d9f-49f9-9e9c-d783306fb289-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92469482-9d9f-49f9-9e9c-d783306fb289" (UID: "92469482-9d9f-49f9-9e9c-d783306fb289"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:07:08.430441 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.430417    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92469482-9d9f-49f9-9e9c-d783306fb289-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:08.431834 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.431814    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92469482-9d9f-49f9-9e9c-d783306fb289-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92469482-9d9f-49f9-9e9c-d783306fb289" (UID: "92469482-9d9f-49f9-9e9c-d783306fb289"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:07:08.431936 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.431917    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92469482-9d9f-49f9-9e9c-d783306fb289-kube-api-access-wdrhd" (OuterVolumeSpecName: "kube-api-access-wdrhd") pod "92469482-9d9f-49f9-9e9c-d783306fb289" (UID: "92469482-9d9f-49f9-9e9c-d783306fb289"). InnerVolumeSpecName "kube-api-access-wdrhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:07:08.531317 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.531289    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdrhd\" (UniqueName: \"kubernetes.io/projected/92469482-9d9f-49f9-9e9c-d783306fb289-kube-api-access-wdrhd\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:08.531317 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.531314    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92469482-9d9f-49f9-9e9c-d783306fb289-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:08.830049 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.830020    2576 generic.go:356] "Generic (PLEG): container finished" podID="92469482-9d9f-49f9-9e9c-d783306fb289" containerID="6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3" exitCode=0
Jul 15 13:07:08.830182 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.830067    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"
Jul 15 13:07:08.830182 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.830069    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t" event={"ID":"92469482-9d9f-49f9-9e9c-d783306fb289","Type":"ContainerDied","Data":"6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3"}
Jul 15 13:07:08.830182 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.830133    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t" event={"ID":"92469482-9d9f-49f9-9e9c-d783306fb289","Type":"ContainerDied","Data":"aa613828a79ca524b01f55be585a5570efa5c1394da96f9b10720bda59f42305"}
Jul 15 13:07:08.830182 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.830163    2576 scope.go:117] "RemoveContainer" containerID="6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3"
Jul 15 13:07:08.839588 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.839570    2576 scope.go:117] "RemoveContainer" containerID="6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3"
Jul 15 13:07:08.839853 ip-10-0-135-6 kubenswrapper[2576]: E0715 13:07:08.839836    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3\": container with ID starting with 6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3 not found: ID does not exist" containerID="6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3"
Jul 15 13:07:08.839918 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.839867    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3"} err="failed to get container status \"6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3\": rpc error: code = NotFound desc = could not find container \"6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3\": container with ID starting with 6e1f5deea005f0cedb3f25b1ef33fd6e68473ee8d5baab044e7dffe1802b2af3 not found: ID does not exist"
Jul 15 13:07:08.852761 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.852740    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"]
Jul 15 13:07:08.858179 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:08.858160    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-3af497-6d8474c9d-hzt5t"]
Jul 15 13:07:09.572081 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:09.572041    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92469482-9d9f-49f9-9e9c-d783306fb289" path="/var/lib/kubelet/pods/92469482-9d9f-49f9-9e9c-d783306fb289/volumes"
Jul 15 13:07:34.002578 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.002526    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"]
Jul 15 13:07:34.003040 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.002790    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" podUID="7efb5521-d788-4c31-a53a-8b51c96e1995" containerName="maas-api" containerID="cri-o://a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0" gracePeriod=30
Jul 15 13:07:34.260354 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.260301    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 13:07:34.343024 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.342998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7efb5521-d788-4c31-a53a-8b51c96e1995-openshift-service-ca\") pod \"7efb5521-d788-4c31-a53a-8b51c96e1995\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") "
Jul 15 13:07:34.343202 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.343087    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls\") pod \"7efb5521-d788-4c31-a53a-8b51c96e1995\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") "
Jul 15 13:07:34.343202 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.343113    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26jnm\" (UniqueName: \"kubernetes.io/projected/7efb5521-d788-4c31-a53a-8b51c96e1995-kube-api-access-26jnm\") pod \"7efb5521-d788-4c31-a53a-8b51c96e1995\" (UID: \"7efb5521-d788-4c31-a53a-8b51c96e1995\") "
Jul 15 13:07:34.343362 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.343340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7efb5521-d788-4c31-a53a-8b51c96e1995-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7efb5521-d788-4c31-a53a-8b51c96e1995" (UID: "7efb5521-d788-4c31-a53a-8b51c96e1995"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:07:34.345020 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.344995    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7efb5521-d788-4c31-a53a-8b51c96e1995-kube-api-access-26jnm" (OuterVolumeSpecName: "kube-api-access-26jnm") pod "7efb5521-d788-4c31-a53a-8b51c96e1995" (UID: "7efb5521-d788-4c31-a53a-8b51c96e1995"). InnerVolumeSpecName "kube-api-access-26jnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:07:34.345115 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.345084    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7efb5521-d788-4c31-a53a-8b51c96e1995" (UID: "7efb5521-d788-4c31-a53a-8b51c96e1995"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 15 13:07:34.444048 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.444023    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7efb5521-d788-4c31-a53a-8b51c96e1995-openshift-service-ca\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:34.444048 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.444046    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7efb5521-d788-4c31-a53a-8b51c96e1995-maas-api-tls\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:34.444188 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.444055    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26jnm\" (UniqueName: \"kubernetes.io/projected/7efb5521-d788-4c31-a53a-8b51c96e1995-kube-api-access-26jnm\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:34.957099 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.957064    2576 generic.go:356] "Generic (PLEG): container finished" podID="7efb5521-d788-4c31-a53a-8b51c96e1995" containerID="a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0" exitCode=0
Jul 15 13:07:34.957277 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.957106    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"
Jul 15 13:07:34.957277 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.957151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" event={"ID":"7efb5521-d788-4c31-a53a-8b51c96e1995","Type":"ContainerDied","Data":"a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0"}
Jul 15 13:07:34.957277 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.957204    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls" event={"ID":"7efb5521-d788-4c31-a53a-8b51c96e1995","Type":"ContainerDied","Data":"c9f8376ec0dec1994780c773b1c573215f37e853009a2b6dc362f0e3d3917aec"}
Jul 15 13:07:34.957277 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.957226    2576 scope.go:117] "RemoveContainer" containerID="a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0"
Jul 15 13:07:34.966742 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.966720    2576 scope.go:117] "RemoveContainer" containerID="a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0"
Jul 15 13:07:34.966986 ip-10-0-135-6 kubenswrapper[2576]: E0715 13:07:34.966965    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0\": container with ID starting with a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0 not found: ID does not exist" containerID="a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0"
Jul 15 13:07:34.967074 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.966993    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0"} err="failed to get container status \"a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0\": rpc error: code = NotFound desc = could not find container \"a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0\": container with ID starting with a03d1721efb55c64c1d5e2c3de54b49b22ba14b5605c255a890a64a4ea38e1e0 not found: ID does not exist"
Jul 15 13:07:34.979679 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.979658    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"]
Jul 15 13:07:34.984381 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:34.984362    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-70ed7b-748578fcbd-xmmls"]
Jul 15 13:07:35.572130 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:35.572099    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7efb5521-d788-4c31-a53a-8b51c96e1995" path="/var/lib/kubelet/pods/7efb5521-d788-4c31-a53a-8b51c96e1995/volumes"
Jul 15 13:07:49.253727 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:49.253696    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"]
Jul 15 13:07:49.254138 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:49.253893    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy" containerID="cri-o://f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d" gracePeriod=30
Jul 15 13:07:54.509092 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.509062    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 13:07:54.516973 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.516915    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-credential-socket\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.516973 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.516941    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wwcw\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-kube-api-access-7wwcw\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.516973 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.516959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-data\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.516973 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.516976    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-podinfo\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.517246 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.516998    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-token\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.517246 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517188    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:07:54.517246 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517206    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-socket\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.517246 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517240    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-envoy\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.517460 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517270    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-certs\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.517460 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517349    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/952fb2d7-1b83-461a-93d3-a414d6085c8e-istiod-ca-cert\") pod \"952fb2d7-1b83-461a-93d3-a414d6085c8e\" (UID: \"952fb2d7-1b83-461a-93d3-a414d6085c8e\") "
Jul 15 13:07:54.517460 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517371    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:07:54.517647 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517550    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:07:54.517765 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517726    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-data" (OuterVolumeSpecName: "istio-data") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:07:54.517824 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517768    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/952fb2d7-1b83-461a-93d3-a414d6085c8e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 15 13:07:54.518000 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517971    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/952fb2d7-1b83-461a-93d3-a414d6085c8e-istiod-ca-cert\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.518000 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.517997    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-credential-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.518130 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.518011    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-data\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.518130 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.518024    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-socket\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.518130 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.518036    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-workload-certs\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.519183 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.519153    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:07:54.519280 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.519254    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 15 13:07:54.519759 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.519742    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 15 13:07:54.519830 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.519817    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-kube-api-access-7wwcw" (OuterVolumeSpecName: "kube-api-access-7wwcw") pod "952fb2d7-1b83-461a-93d3-a414d6085c8e" (UID: "952fb2d7-1b83-461a-93d3-a414d6085c8e"). InnerVolumeSpecName "kube-api-access-7wwcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 15 13:07:54.618786 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.618747    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wwcw\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-kube-api-access-7wwcw\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.618786 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.618783    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-podinfo\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.618786 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.618793    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-token\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:54.618786 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:54.618800    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/952fb2d7-1b83-461a-93d3-a414d6085c8e-istio-envoy\") on node \"ip-10-0-135-6.ec2.internal\" DevicePath \"\""
Jul 15 13:07:55.055828 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.055793    2576 generic.go:356] "Generic (PLEG): container finished" podID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerID="f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d" exitCode=0
Jul 15 13:07:55.055985 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.055850    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"
Jul 15 13:07:55.055985 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.055873    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" event={"ID":"952fb2d7-1b83-461a-93d3-a414d6085c8e","Type":"ContainerDied","Data":"f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d"}
Jul 15 13:07:55.055985 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.055928    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh" event={"ID":"952fb2d7-1b83-461a-93d3-a414d6085c8e","Type":"ContainerDied","Data":"1a756d3a2278309d81109453e4bef0c0327304a7881a6ef147246e2ebc11a5bb"}
Jul 15 13:07:55.055985 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.055951    2576 scope.go:117] "RemoveContainer" containerID="f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d"
Jul 15 13:07:55.065391 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.065370    2576 scope.go:117] "RemoveContainer" containerID="22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"
Jul 15 13:07:55.072929 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.072913    2576 scope.go:117] "RemoveContainer" containerID="f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d"
Jul 15 13:07:55.073186 ip-10-0-135-6 kubenswrapper[2576]: E0715 13:07:55.073166    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d\": container with ID starting with f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d not found: ID does not exist" containerID="f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d"
Jul 15 13:07:55.073231 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.073196    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d"} err="failed to get container status \"f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d\": rpc error: code = NotFound desc = could not find container \"f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d\": container with ID starting with f99489db91454e1725fbe4bb84d5220d9e59dfac02e611bb303d6cffe4ccec6d not found: ID does not exist"
Jul 15 13:07:55.073231 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.073212    2576 scope.go:117] "RemoveContainer" containerID="22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"
Jul 15 13:07:55.073430 ip-10-0-135-6 kubenswrapper[2576]: E0715 13:07:55.073414    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726\": container with ID starting with 22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726 not found: ID does not exist" containerID="22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"
Jul 15 13:07:55.073476 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.073436    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726"} err="failed to get container status \"22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726\": rpc error: code = NotFound desc = could not find container \"22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726\": container with ID starting with 22f1b4d772c4c1623e6437683822f673f958e9e1fb564d85b4c5d0dd008cb726 not found: ID does not exist"
Jul 15 13:07:55.083476 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.083455    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"]
Jul 15 13:07:55.090022 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.090000    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-70ed7b-openshift-default-64cf66ffff-d4lzh"]
Jul 15 13:07:55.511640 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.511612    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-679db_57dc7b45-6bde-44ce-ab43-7f38c5a19fae/manager/0.log"
Jul 15 13:07:55.572470 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.572444    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" path="/var/lib/kubelet/pods/952fb2d7-1b83-461a-93d3-a414d6085c8e/volumes"
Jul 15 13:07:55.630589 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.630563    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74fcfc6ff5-jr8kd_fdd0db81-5abc-41a8-9abc-5d01af1501cb/manager/0.log"
Jul 15 13:07:55.975760 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.975738    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-b7lvd_bb2ac1c9-0b49-4af4-b9f9-90a90f46349a/copy-manifests/0.log"
Jul 15 13:07:55.989652 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:55.989632    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-b7lvd_bb2ac1c9-0b49-4af4-b9f9-90a90f46349a/manager/0.log"
Jul 15 13:07:56.589801 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:56.589773    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-848c5c555b-bnrb7_3e7801fd-6c76-4895-9307-667d6fcd92fa/maas-api/0.log"
Jul 15 13:07:56.707115 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:56.707092    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735325-sp5p4_9fda6ad1-c204-4f28-81e5-61c149063bf8/cleanup/0.log"
Jul 15 13:07:58.252069 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:58.252041    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-f5kf7_894eecde-850c-4947-b9be-95d7a61051fa/postgres/0.log"
Jul 15 13:07:59.759395 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:07:59.759365    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rhshn_66f97a63-1864-41a9-835f-9007bdb83527/manager/0.log"
Jul 15 13:08:01.179364 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:01.179335    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/1.log"
Jul 15 13:08:01.297738 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:01.297713    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-w8vs9_74d4e7a1-6b77-492b-8d7c-a1aaaef49046/payload-pre-processing/0.log"
Jul 15 13:08:01.529785 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:01.529720    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-595544f54d-bxnpp_b5a3b99b-1972-4d70-9f3b-3109e79e1c5c/router/0.log"
Jul 15 13:08:01.874624 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:01.874554    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-745xh_61aacc84-4e1a-460f-b489-0854e26cfe91/main/0.log"
Jul 15 13:08:01.985397 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:01.985370    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-xc4np_d1d611ca-1c1c-4500-9c37-aeb00b3675f7/main/0.log"
Jul 15 13:08:02.216242 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:02.216223    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbxf7hq_8f87f326-b03f-45ec-b7ba-1dfa66637fa9/main/0.log"
Jul 15 13:08:11.097502 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:11.097473    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7kfpt_ac68a54c-2842-44d0-b0a5-5fa81fd0e1e6/global-pull-secret-syncer/0.log"
Jul 15 13:08:11.272232 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:11.272206    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wswpt_33b02fa8-a98a-4ec5-9e7d-98f2edfc0d77/konnectivity-agent/0.log"
Jul 15 13:08:11.299333 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:11.299308    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-6.ec2.internal_bd7265a5e5fed270cfc17e07cb6ae380/haproxy/0.log"
Jul 15 13:08:16.038433 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:16.038402    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rhshn_66f97a63-1864-41a9-835f-9007bdb83527/manager/0.log"
Jul 15 13:08:18.347198 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:18.347172    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-qnt9h_17d63f31-cbbf-43c0-9ef4-bb82b110f4ce/cluster-monitoring-operator/0.log"
Jul 15 13:08:18.725723 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:18.725694    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zkfv5_1c2ba17b-d828-4945-92f6-556aac9d4ac2/node-exporter/0.log"
Jul 15 13:08:18.753045 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:18.753010    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zkfv5_1c2ba17b-d828-4945-92f6-556aac9d4ac2/kube-rbac-proxy/0.log"
Jul 15 13:08:18.780453 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:18.780425    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zkfv5_1c2ba17b-d828-4945-92f6-556aac9d4ac2/init-textfile/0.log"
Jul 15 13:08:20.567930 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.567903    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-m78dh_b3b94358-208c-4e52-9d82-b7de8a12e86d/networking-console-plugin/0.log"
Jul 15 13:08:20.917092 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917062    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"]
Jul 15 13:08:20.917446 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917434    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy"
Jul 15 13:08:20.917489 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917448    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy"
Jul 15 13:08:20.917489 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917458    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7efb5521-d788-4c31-a53a-8b51c96e1995" containerName="maas-api"
Jul 15 13:08:20.917489 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917465    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7efb5521-d788-4c31-a53a-8b51c96e1995" containerName="maas-api"
Jul 15 13:08:20.917489 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917478    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92469482-9d9f-49f9-9e9c-d783306fb289" containerName="maas-api"
Jul 15 13:08:20.917489 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917483    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="92469482-9d9f-49f9-9e9c-d783306fb289" containerName="maas-api"
Jul 15 13:08:20.917489 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917491    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy"
Jul 15 13:08:20.917690 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917496    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy"
Jul 15 13:08:20.917690 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917558    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7efb5521-d788-4c31-a53a-8b51c96e1995" containerName="maas-api"
Jul 15 13:08:20.917690 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917570    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="952fb2d7-1b83-461a-93d3-a414d6085c8e" containerName="istio-proxy"
Jul 15 13:08:20.917690 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.917576    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="92469482-9d9f-49f9-9e9c-d783306fb289" containerName="maas-api"
Jul 15 13:08:20.922473 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.922455    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:20.924813 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.924782    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mxws\"/\"openshift-service-ca.crt\""
Jul 15 13:08:20.925647 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.925622    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mxws\"/\"kube-root-ca.crt\""
Jul 15 13:08:20.925801 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.925701    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8mxws\"/\"default-dockercfg-7bjhq\""
Jul 15 13:08:20.930683 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:20.930662    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"]
Jul 15 13:08:21.023290 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.023264    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-proc\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.023290 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.023291    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-lib-modules\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.023478 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.023314    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-podres\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.023478 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.023403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-sys\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.023615 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.023515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpn6t\" (UniqueName: \"kubernetes.io/projected/546dd18a-f78e-44cb-9a0a-70b6a665e264-kube-api-access-qpn6t\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124553 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-proc\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124553 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124549    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-lib-modules\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124734 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-podres\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124734 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124617    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-sys\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124734 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-proc\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124734 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124678    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-lib-modules\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124734 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124699    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpn6t\" (UniqueName: \"kubernetes.io/projected/546dd18a-f78e-44cb-9a0a-70b6a665e264-kube-api-access-qpn6t\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124734 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124727    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-podres\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.124977 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.124699    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/546dd18a-f78e-44cb-9a0a-70b6a665e264-sys\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.133187 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.133161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpn6t\" (UniqueName: \"kubernetes.io/projected/546dd18a-f78e-44cb-9a0a-70b6a665e264-kube-api-access-qpn6t\") pod \"perf-node-gather-daemonset-mwqn4\" (UID: \"546dd18a-f78e-44cb-9a0a-70b6a665e264\") " pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.232516 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.232459    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:21.260830 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.260804    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-dbnzx_625c565a-4792-44ae-bc4b-9bf8004d9a52/console-operator/0.log"
Jul 15 13:08:21.359225 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:21.359200    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"]
Jul 15 13:08:21.361625 ip-10-0-135-6 kubenswrapper[2576]: W0715 13:08:21.361599    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod546dd18a_f78e_44cb_9a0a_70b6a665e264.slice/crio-4417cbff48bf6af88c69a767be841b87b7b9dd2cf9f75c25af469e4e9ef4ba4e WatchSource:0}: Error finding container 4417cbff48bf6af88c69a767be841b87b7b9dd2cf9f75c25af469e4e9ef4ba4e: Status 404 returned error can't find the container with id 4417cbff48bf6af88c69a767be841b87b7b9dd2cf9f75c25af469e4e9ef4ba4e
Jul 15 13:08:22.186377 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:22.186346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4" event={"ID":"546dd18a-f78e-44cb-9a0a-70b6a665e264","Type":"ContainerStarted","Data":"66ef6900194cc7665c7ade97fa1186d4bde933dd0d6b0a9af0e458f6028d3f4b"}
Jul 15 13:08:22.186737 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:22.186402    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4" event={"ID":"546dd18a-f78e-44cb-9a0a-70b6a665e264","Type":"ContainerStarted","Data":"4417cbff48bf6af88c69a767be841b87b7b9dd2cf9f75c25af469e4e9ef4ba4e"}
Jul 15 13:08:22.186737 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:22.186430    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:22.204707 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:22.204653    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4" podStartSLOduration=2.204634193 podStartE2EDuration="2.204634193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 13:08:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 13:08:22.20220918 +0000 UTC m=+2891.078932214" watchObservedRunningTime="2026-07-15 13:08:22.204634193 +0000 UTC m=+2891.081357199"
Jul 15 13:08:22.480303 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:22.480230    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-vqh5z_5b2334d3-157e-4349-b2dc-a93e36144b49/volume-data-source-validator/0.log"
Jul 15 13:08:23.667066 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:23.667042    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zgxhc_95aa8e11-cade-4b79-bd27-10d2459910a4/dns/0.log"
Jul 15 13:08:23.690910 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:23.690886    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zgxhc_95aa8e11-cade-4b79-bd27-10d2459910a4/kube-rbac-proxy/0.log"
Jul 15 13:08:23.745504 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:23.745482    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hgvm9_3d6d2951-2153-434e-b87f-bff9e54fb088/dns-node-resolver/0.log"
Jul 15 13:08:24.390926 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:24.390902    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fjhkd_15a5102c-c886-4f37-ba0c-2bb0f653fa09/node-ca/0.log"
Jul 15 13:08:25.661599 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:25.661571    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/0.log"
Jul 15 13:08:25.689022 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:25.688993    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/1.log"
Jul 15 13:08:25.718333 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:25.718306    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-w8vs9_74d4e7a1-6b77-492b-8d7c-a1aaaef49046/payload-pre-processing/0.log"
Jul 15 13:08:25.774623 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:25.774603    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-595544f54d-bxnpp_b5a3b99b-1972-4d70-9f3b-3109e79e1c5c/router/0.log"
Jul 15 13:08:26.514357 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:26.514332    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pwmnc_a560a9c7-677a-4ad2-9b65-1e15843b171d/serve-healthcheck-canary/0.log"
Jul 15 13:08:27.097799 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:27.097763    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-zcxwn_c5eb7b44-4536-4cc3-8ef0-97a8def2a583/insights-operator/0.log"
Jul 15 13:08:27.195031 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:27.195009    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtxtm_b555f300-48b7-4ff5-ba2b-4c45bec3e8df/kube-rbac-proxy/0.log"
Jul 15 13:08:27.219608 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:27.219583    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtxtm_b555f300-48b7-4ff5-ba2b-4c45bec3e8df/exporter/0.log"
Jul 15 13:08:27.245382 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:27.245361    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mtxtm_b555f300-48b7-4ff5-ba2b-4c45bec3e8df/extractor/0.log"
Jul 15 13:08:28.201131 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:28.201105    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8mxws/perf-node-gather-daemonset-mwqn4"
Jul 15 13:08:29.845151 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:29.845123    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-679db_57dc7b45-6bde-44ce-ab43-7f38c5a19fae/manager/0.log"
Jul 15 13:08:29.940870 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:29.940837    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74fcfc6ff5-jr8kd_fdd0db81-5abc-41a8-9abc-5d01af1501cb/manager/0.log"
Jul 15 13:08:30.117758 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:30.117731    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-b7lvd_bb2ac1c9-0b49-4af4-b9f9-90a90f46349a/manager/0.log"
Jul 15 13:08:30.142096 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:30.142079    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f56ddd455-b7lvd_bb2ac1c9-0b49-4af4-b9f9-90a90f46349a/copy-manifests/0.log"
Jul 15 13:08:31.613090 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:31.613066    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-595c7f4b4c-5nqsk_f29a74b3-777c-4c23-8fd7-339bcb8f2df1/manager/0.log"
Jul 15 13:08:31.698200 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:31.698169    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/0.log"
Jul 15 13:08:31.723402 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:31.723374    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/1.log"
Jul 15 13:08:31.756732 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:31.756710    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-w8vs9_74d4e7a1-6b77-492b-8d7c-a1aaaef49046/payload-pre-processing/0.log"
Jul 15 13:08:31.801378 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:31.801355    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-595544f54d-bxnpp_b5a3b99b-1972-4d70-9f3b-3109e79e1c5c/router/0.log"
Jul 15 13:08:33.290102 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:33.290075    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/0.log"
Jul 15 13:08:33.316750 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:33.316723    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-kfflk_f6e88f17-42ee-47aa-b377-572cee4dcb0f/istio-proxy/1.log"
Jul 15 13:08:33.341524 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:33.341496    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-w8vs9_74d4e7a1-6b77-492b-8d7c-a1aaaef49046/payload-pre-processing/0.log"
Jul 15 13:08:33.385312 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:33.385281    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-595544f54d-bxnpp_b5a3b99b-1972-4d70-9f3b-3109e79e1c5c/router/0.log"
Jul 15 13:08:36.984597 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:36.984570    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-j4mvm_b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b/migrator/0.log"
Jul 15 13:08:37.008673 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:37.008648    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-j4mvm_b34d1a1c-3647-4c4e-a47f-f23bb2b1e18b/graceful-termination/0.log"
Jul 15 13:08:37.443707 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:37.443677    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-fmzhh_c74f7ff8-ae45-4bf9-ad15-59abca5df8b2/kube-storage-version-migrator-operator/1.log"
Jul 15 13:08:37.444401 ip-10-0-135-6 kubenswrapper[2576]: I0715 13:08:37.444383    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-fmzhh_c74f7ff8-ae45-4bf9-ad15-59abca5df8b2/kube-storage-version-migrator-operator/0.log"